Revelando as relações entre métodos explicativos e desempenho de agentes de aprendizado por reforço

Abstract

The advancement of technology and artificial intelligence has driven the use of reinforcement learning algorithms in areas such as education and video games. Despite the success of these applications, it is still unclear what factors agents learn that ensure their high performance. Understanding how these characteristics impact results is essential for increasing user confidence and identifying failures, especially in critical applications. This work investigates the influence of performance and robustness of machine learning models on the features that agents observe when solving problems. The study analyzes whether changes in explainability are noticeable among agents with different levels of robustness, using Explainable Artificial Intelligence methods and whether these changes can predict agents’ performance in environments outside the training set. Two experiments were carried out to conduct the analysis. In the first experiment, agents with different levels of robustness were analyzed using methods such as Grad-CAM, Integrated Gradients, SHAP, and LIME to generate explanations. The impact of robustness on the agents’ explainability was evaluated through qualitative and quantitative analyses. In the second experiment, variations in models and scenarios were compared to identify similarities and correlations with the rewards obtained by each agent, aiming to understand the relationship between the agent’s performance and the observed characteristics. The results show significant variations in explainability among agents with different levels of robustness, which can be used to predict model performance in unknown environments. It was also observed that agent performance is correlated with the similarity of explanations: agents whose explanations resemble those of successful models are more likely to achieve high performance. It is concluded, therefore, that the robustness and performance of an agent are linked to the learned features for solving a problem.