Pipeline de processamento de dados de dispositivos móveis em transporte coletivo urbano

Abstract

Every day, new products or services are created seeking to meet an existing need, or simply improve the current form of the solution offered. As a result, there is an increase in the volumes of data generated by these systems. Mobile devices like smartphones have become ubiquitous in everyday life. However, these mobile devices have computational limitations, so it is common for modern systems to use cloud computing for data analysis tasks generated by the devices. The complete flow of data analysis goes through several steps such as: collection, formatting, secure transmission, storage, processing, analysis, and availability of results. The pipeline is a software design pattern that provides the ability to build and execute a sequence of operations. Thus, it is possible to establish tools and solutions for all phases of the flow. This work presents a proposal for a solution for processing this data through the creation of a pipeline for processing data streams for mobile applications. The solution was made using tools like Apache Storm(Stream Processing Framework), Apache Kafka(Stream Processing Framework), MongoDB(Document Oriented Database), Express.js(Framework for Node.js) for the part and Flutter(Framework for building cross-platform applications) for building mobile applications. With the proposed pipeline, a test system was created that enables the notification of passengers about the proximity of an urban transport vehicle of interest. The competence of the pipeline for the construction of this operating pattern was demonstrated. A series of manual tests were carried out for the proposed scenario of use, however, more robust tests need to be performed in order to compare performance with other possible solutions and scalability.