Além da cobertura de código: avaliando a qualidade de testes em componentes Symfony

Abstract

This work investigates the efficiency of mutation testing as a complement to code coverage in Symfony components, proposing to evaluate the quality of test suites. The research analyzes 31 open source repositories with high coverage (above 80%), applying the Infection tool for mutation testing. Results reveal a moderate correlation (R 2 = 0.386) between code coverage and MSI, showing that coverage alone is not a sufficient indicator of test effectiveness. It was observed that even repositories with high coverage (90-99%) presented significant variations in MSI (50-100%), with eight repositories showing discrepancies greater than 20 percentage points between metrics. Notable cases include "password-hasher" (89.5% coverage and 58% MSI) and "templating" (81.25% coverage and 51% MSI). The analysis of surviving mutants demonstrated a negative correlation with coverage, but with relevant exceptions, such as the "dependencyinjection" repository (91.30% coverage and 2,569 surviving mutants). The study concludes that mutation testing constitutes an effective technique to complement coverage analysis, contributing to continuous improvement of tests in open source projects and highlighting the need for a multifaceted approach in evaluating software quality.