Controle de movimento de personagens fisicamente simulados usando funções de recompensa

Abstract

Generating natural movements for physics-based articulated characters is challenging mainly because adjusting the visual quality of the movement often compromises the basic functioning of the controller, as the control parameters generally have a non-intuitive relationship with the resulting animation. Deep Reinforcement Learning (DRL) has been a recently explored approach to treat the control problem in such structures, in which a neural network is used to deal with this relationship between control inputs and information outputs to be used by the actuators. Further, the definition of an appropriate reward is necessary to guide the learning process. Although the learning process takes a long time, a major advantage of the DRL method is that the trained network works in real-time. This work, therefore, proposes adjustments in the reward function and in the network input and output information to provide the animator with a greater degree of control over the resulting movements. That control is explored both in the training phase and during the simulation in real-time. The proposed reward terms were adapted for two characters with different morphologies and proved to be able to clearly differentiate interesting types of locomotion, such as running and jumping. Adjustments to network input and output information allowed real-time speed control and the possible imposition of symmetry on the character’s movement. Experiments simulating interactions with the environment, such as objects being thrown at the character and modifications causing irregularities in the terrain, showed the robustness of the control obtained using DRL.