Autoeval-ct: um artefato de avaliação semi-automatizada do pensamento computacional

Abstract

This work presents a doctoral thesis whose objective is the development and analysis of an evaluative approach and an artifact designed to assist high school teachers in assessing students’ learning of Computational Thinking (CT). The teaching and assessment of CT have become fundamental aspects of contemporary education, especially after its inclusion in the BNCC. However, the assessment of this competence still facessignificant challenges, particularly due to the scarcity of proposals and tools that enable an efficient and systematic process. To address this problem, the development and evaluation of AutoEval-CT is proposed, an artifact capable of automatically analyzing programming code produced by students, identifying evidence of CT in the skills of abstraction, decomposition, pattern recognition, and algorithmic thinking. The artifact employs LLM models to carry out this analysis and generate detailed reports that support teachers in the assessment process. The evaluation of the tool was conducted according to the principles of DSR, in multiple stages, including tests with experts, comparisons between human and semi-automated assessments, analyses of different LLM models, and a controlled experiment with a high school class. The central hypothesis of this research is that the adoption of a hierarchical quali-quantitative evaluative approach, together with an artifact based on the semi-automated analysis of programming code, provides support for teachers in assessing CT in high school students. It is expected that the results of this research will contribute to supporting high school teachers by providing an innovative solution for the assessment of CT, as well as offering empirical evidence for the development of future studies.