Análise dos registros de conversação de um protótipo de chatbot desenvolvido para pessoas com dor lombar

Abstract

Introduction: Low back pain (LBP) is a condition with a high global and challenging prevalence. Chatbots have the potential to help with DL self-management, but it is necessary to analyze and improve the dialogues between chatbots and users for a better usability experience. Researchers developed a chatbot prototype for DL self-management, seeking to explore and improve conversational capabilities through dialogue analysis. Objective: analyze the IVO chatbot dialogues to improve its usability and response quality. Methods: The developed chatbot operates via Facebook Messenger and uses a dialogue tree. User interests are identified through Natural Language Processing (NLP), a technique in which the virtual assistant identifies keywords and linguistic patterns to interpret the intentions expressed by the user, to ensure useful responses. We collected data from the initial phase of usability testing with participants suffering from LBP for more than 3 months, in person or remotely. We analyze dialogue paths, accessed topics, unanswered topics, bugs, interaction time, and categorize the quality of the conversation to calculate the fallback rate. We also record statistics such as total messages and prototype usage time. Results: The results of the analysis of conversation records between the chatbot and users point to topics of greatest interest to users, such as "reducing and controlling pain" and "general exercises". Regarding the quality of the interaction, 52.2% of requests were considered "Resolved" and 17.7% "Partially Resolved", in addition to a fallback rate of 29.9%. However, we also observed that approximately 13.3% of requests were not resolved due to a lack of response from the chatbot regarding specific topics. Conclusion: The paths taken by users of the chatbot developed for the management of LBP showed access to specific paths, focusing on practical themes of treatment and pain interruption. Although 52.2% of interactions were successfully resolved, the chatbot's responsiveness can be improved, and it is important to manage user expectations. The most accessed topics highlight the relevance of clear guidance on the treatment and self-management of LBP, guiding future steps in the development of the virtual assistant.