Exploração de ambientes virtuais em jogos: uma abordagem baseada em aprendizado por reforço

Abstract

This work explores the reinforcement learning technique based on the proximal policy optimization (PPO) method to train a machine learning (ML) model to play a virtual game. This method is widely recognized for its efficiency and stability in reinforcement learning problems, especially in scenarios where the rewards may be unknown or difficult to define. Reinforcement learning has been used in a variety of areas, including autonomous driving, news recommendation, natural language processing, game testing and creation, traffic control, and drug discovery. The game Legend of Zelda: Link’s Awakening, which runs on Nintendo’s portable console called Game Boy released in 1993, was chosen to serve as the training environment for the model. The chosen approach uses the Pyboy library to emulate the game and run multiple instances in parallel and the Gymnasium library to create the reinforcement learning model. This model was chosen because it has a low training cost and performs well in deterministic, high-cardinality environments. The model also provides a balance between ease of implementation, sample complexity, and ease of adjustment. PPO is designed to update the policy incrementally, ensuring that changes are not too drastic and that the updated policy remains close to the previous policy. The main goal is to maximize expected rewards over time by continuously adjusting the agent’s actions based on the rewards received from the environment. Exploration is a crucial aspect of reinforcement learning, as it allows the agent to discover new strategies and actions that may lead to greater rewards in the future. This paper discusses the algorithm and methodology used to train the model.