BuriedBrains: um ambiente multiagente inspirado em roguelike para avaliação de memória e tomada de decisão

Abstract

This work presents the development and validation of BuriedBrains, a multi-agent simulation environment based on procedural generation and partial observability, structured through hybrid dynamic graphs. The primary objective is to measure the performance of Reinforcement Learning (RL) agents in solving long-term planning and generalization problems under conditions of permadeath and resource constraints. The environment’s topology, inspired by roguelike mechanics — specifically the title Buriedbornes (Nussygame, 2016) —, utilizes Directed Acyclic Graphs (DAGs) for Progression Zones and cyclic graphs for Sanctuaries. These structures demand inventory and skill management, as state transitions result in persistent changes to variables such as Karma and the agent’s survival probability. The methodology consisted of a comparative study to isolate the impact of temporal recurrence, contrasting a reactive architecture (PPO) with a recurrent one (LSTM). Both models were refined through Hyperparameter Optimization (HPO) and integrated with Self-Attention mechanisms. Data suggests that the characteristics of BuriedBrains restrict the performance of reactive models under the tested conditions, as reflected by PPO’s stabilization at a 65% Explained Variance (EV) plateau, which remained constant despite increases in network complexity or sampling horizon. The recurrent agent demonstrated 24% superior performance in HPO tests and reached 80% explained variance. Results suggest the architecture was capable of sustaining policies focused on attribute specialization and exhibiting multi-agent interaction patterns. Visualization-based analysis identified convergence toward reciprocal non-aggression states, where memory enabled the optimization of episode longevity over zero-sum interactions. BuriedBrains thus serves as a benchmark for evaluating long-term temporal dependency resolution and decision-making in autonomous systems.