Challenges and opportunities on software testing teaching

Abstract

The investigation into the use of industry tools for test generation oracles, specifically EvoSuite, for pedagogical support is an area that requires further study. In this context, an experiment was designed with the objective of evaluating the impact of EvoSuite on the teaching and learning process of software testing in computing courses. The experiment involved the creation of test cases, with participants divided into two groups: the control group, which received only the source code and documentation, and the experimental group, which additionally received a suite of tests automatically generated by EvoSuite. The students’ perceptions were assessed through questionnaires administered at three distinct stages: at the beginning, during, and at the end of the experiment. The results indicated that the use of EvoSuite enabled the discovery of new test scenarios and innovative approaches, suggesting that automatic test generation tools can significantly contribute to improving students’ testing skills and the quality of the tests generated. Subsequently, a second study was conducted with the aim of developing a tool that integrates large language models, such as ChatGPT, with an online IDE, providing students with a practical platform to enhance their software testing skills. The proposal also included the use of PiTest for mutation analysis, aiming to provide an additional layer of evaluation of the quality of the tests developed by the students. This work comprises two studies: the first focused on the pedagogical use of EvoSuite, where the results showed that the tool aids in identifying test scenarios not initially foreseen, and the second focused on the integration of artificial intelligence into educational tools, providing a practical and interactive environment for students. Both studies highlight the importance of adopting automated practices, such as unit testing and continuous integration, while also emphasizing the central role of interpersonal skills, such as communication and problem-solving, in the training of qualified professionals. The conclusions of these studies offer relevant implications for educators and students, suggesting that the combination of practical learning, automation, and interpersonal skills can significantly enhance software testing education.