Uma abordagem de teste baseada em guias estruturados para avaliar características da Internet das Coisas

Abstract

Internet of Things (IoT) applications are characterized by the use of smart objects that are connected to the Internet to provide different types of services. These smart objects typically generate data that needs to be stored and analyzed to contribute to decision making (whether immediate or not). In this context, such applications may use different communication protocols, requiring high performance and, in some cases, real-time responses. These requirements bring new testing challenges and the need for specific approaches, for example, the detection of performance gaps between heterogeneous IoT devices that process a large amount of data and, under certain conditions, must have their resources optimized. Thus, this master’s thesis defines a testing approach to evaluate IoT applications characteristics based on structured guides. To build this approach, a literature review was conducted to identify guides, guidelines and technical standards, which are used in IoT application evaluations. In addition, two bug repositories were analyzed to understand the particularities of IoT testing activities. Based on this research, the guide was proposed based on 11 topics, which are: characteristic definition, correlation of characteristics, environment configuration, definition of subcharacteristics, contextualization, abstract test cases, measurements, impact of subcharacteristics, cost-benefit, tool suggestions, and usage example. The guide was instantiated for the Performance characteristic, which is composed of three subcharacteristics: Temporal Behavior, Resource Utilization, and Capacity. To automate the use of the guide, we built a Wiki to support the performance test planning. To evaluate the guide, two evaluations were conducted: (i) an evaluation with the experts to validate the structure and content of the guide; and (ii) a controlled experiment to evaluate the use of the guide in a real IoT application. The results showed that the guide is helpful in evaluating performance aspects of IoT applications. Moreover, the structure of the guide itself provides a systematization of testing activities and can be adapted to evaluate other IoT characteristics. This organization generates more effective results and an easier detection of IoT failures.