Experimentos de transferência neural de estilo em aplicações de visão computacional

Abstract

The objective of this work is the development and prototyping of augmented reality solutions that combine object detection and neural style transfer. Throughout the work, technologies will be explored to address the mentioned challenges, in addition to highlighting the limitations faced during development, such as computational performance, model complexity, and result optimization. A preliminary research was required to survey technologies for object detection, neural style transfer, and augmented reality. For the elaboration and prototyping of the work, technologies such as ARCore, YOLO, convolutional neural networks, Python, PyTorch, and the OpenCV library were utilized. This study presents the development of three prototypes, with the third being provided in two distinct versions: the original and an enhanced version with the implementation of significant performance improvements. As a result of the prototypes, the first one emphasizes object detection and manipulation of the detected object, while the second highlights the integration of object detection and neural style transfer exclusively on the detected objects, preserving the rest of the scene in its original form. Furthermore, the third and final prototype maintains the functionality of the second prototype, but now each class of detected objects is assigned one of the two chosen styles for the work, specifically the works "Self-portrait" by Pablo Picasso and "O Gato" by Romero Britto. Finally, the performance of the developed solutions was considered in certain contexts to verify their effectiveness and adaptability. Notably, the solution demonstrated the ability to run at over 60 frames per second in the first prototype and around 10 frames in the third prototype, using a machine equipped with an RTX 2060 graphics card to execute the solution in both aforementioned cases.