ViSCo (View, Scan, and Control it): uso de visão computacional para descoberta de serviços em ambientes residenciais inteligentes

Abstract

The number of everyday objects (e.g., lamps, doors, and TVs) connected to the Internet is on the ascent and already exceeded 20 billion devices. Some of them offer interaction interfaces through network invocation protocols (e.g., MQTT, CoAP, and REST), allowing users to search and control these devices. However, it is still necessary for the user to find and connect with those smart objects, for example, using a mobile application that uses some service discovery mechanism (e.g., UPnP). Typically, a service discovery protocol is limited to listing all devices that are within range of the network, and that respond to that protocol. However, with a large number of devices, search interfaces tend to suffer from information overload in their results. In this work, we extended the service discovery mechanism of the openHAB Smart Home platform. In our approach, called ViSCo (View, Scan, and Control it), the focus is to aggregate the user’s field of view, captured by the camera of her smartphone, as a smart object search filter. Our approach searches the object of interest combining information obtained using computer vision algorithms to the result of the service discovery mechanism of openHAB. With our approach, we aim to decrease the number of objects returned to the final user and offer a new way to search and control smart devices. During the ViSCo design, we collected users’ opinions about the approach and some real user image examples of objects they would like to control. We’ve trained our model and validate it with Web images and the users’ photos. Results showed 83% and 72.9% of accuracy respectively. Also, we show a Proof-of-Concept of the openHAB mobile application extension to View, Scan, and Control a TV. The final solution was tested by 17 users, remotely, in an environment with virtual devices, where they had used the ViSCo approach to find these devices pointing to real objects in their homes. After the users had the experience to use the ViSCo approach they answered a System Usability Scale (SUS) survey about ViSCo ant the results showed a good level of acceptance, with an average score of 83,97 of SUS.