Avaliação e gestão de cursos de graduação: desenvolvimento de framework baseado em inteligência artificial

Abstract

The institutional evaluation of undergraduate programs in Brazil, regulated by the National Higher Education Evaluation System (SINAES), generates significant volumes of assessment data whose translation into strategic action plans frequently exceeds the analytical capacity of Institutional Self-Assessment Committees (CPA) and course coordinators, compromising the formative function of the system and reinforcing a culture of compliance rather than a culture of quality. This dissertation aims to develop and test a framework based on Generative Artificial Intelligence agents to support the institutional evaluation processes of undergraduate programs, transforming the normative criteria of SINAES into structured diagnoses and actionable strategic recommendations. The research adopts Design Science Research (DSR) as its methodological approach, operationalized through the DSRM model, positioning the study within the improvement quadrant. As methodological contributions, three analytical artifacts are proposed, built through Content Analysis applied to INEP's Undergraduate Course Assessment Instrument: the Conceptual Progression Matrix (MPC), which decomposes the five conceptual levels of each indicator into constitutive elements, progression attributes, and required evidence; the Applicability Conditions Matrix (MCA), which codifies rules determining indicator applicability; and the Evidence Sharing Matrix (MCE), which classifies the scope of documentary artifact sharing. The framework was materialized in the Notacinco platform, a multi-tenant web application, featuring three specialized generative AI agents (Diagnosis, Recommendation, and Synthesis) and a Retrieval-Augmented Generation (RAG) pipeline. Multi-method validation, conducted across seven programs at the State University of Mato Grosso (UNEMAT) with sixty executions, yielded an Overall F1-Score of 0.9582, Diagnostic Accuracy of 1.0000, Structural Consistency Coefficient of 0.9759, and Semantic Consistency Index of 0.9509. It is concluded that the framework contributes to transforming evaluative data into actionable knowledge, supporting course coordinators and CPA members in formulating evidence-based improvement plans aligned with SINAES requirements.