Avaliação da usabilidade do chatbot minhaUFC: uma abordagem baseada em métados de inspeção e feedback de usuários

Abstract

Chatbots have been widely adopted in various environments, including academic settings, as an alternative to facilitate students’ access to institutional information. However, the usability of these systems is not always satisfactory, which compromises both user experience and communication efficiency. In this context, this study aimed to evaluate the usability of the minhaUFC chatbot, a tool used by the Federal University of Ceará. This is a case study with a quali-quantitative approach, in which three evaluation methods were employed: heuristic analysis, the U2CHATBOT checklist, and the SUS questionnaire applied to users. In the heuristic analysis, the highest incidence of problems was related to the heuristics of User Control and Freedom and System Visibility, with notable issues including the absence of buttons to return to the previous menu and failures in the flow of typed questions. These same problems were confirmed by the checklist, which revealed that only 29% of the items were compliant, while 29% presented problems and 17% possible problems, with the most critical categories being User Control and Freedom, Help and Documentation, and Flexibility and Efficiency of Use. The SUS questionnaire obtained an overall average score of 81.5 points, considered good, indicating a positive perception by users regarding the system’s simplicity and reliability. In summary, the evaluation showed that the minhaUFC chatbot has satisfactory usability for simple tasks but requires adjustments to improve navigation, enhance accessibility, and make interactions more fluid and efficient.