In this paper we use the Proximal Policy Optimization (PPO) deep reinforcement learning algorithm to train a Neural Network to control a four-legged robot in simulation. Reinforcement learning in general can learn complex behavior policies from simple state-reward tuples datasets and PPO in particular has proved its effectiveness in solving complex tasks with continuous states and actions. Moreover, since it is model-free, it is general and can adapt to changes in the environment or in the robot itself. The virtual environment used to train the agent was modeled using our physics engine Project Chrono. Chrono can handle non smooth dynamics simulation allowing us to introduce stiff leg-ground contacts and using its Python interface Pychrono it can be interfaced with the Machine Leaning framework TensorFlow with ease. We trained the Neural Network until it learned to control the motor torques, then various policy Neural Network input state choices have been compared.

Training a Four Legged Robot via Deep Reinforcement Learning and Multibody Simulation / Benatti, S.; Tasora, A.; Mangoni, D.. - 53:(2020), pp. 391-398. (Intervento presentato al convegno ECCOMAS Multibody Dynamics 2019 tenutosi a Duisburg, Germany) [10.1007/978-3-030-23132-3_47].

Training a Four Legged Robot via Deep Reinforcement Learning and Multibody Simulation

Benatti S.;Tasora A.;Mangoni D.
2020-01-01

Abstract

In this paper we use the Proximal Policy Optimization (PPO) deep reinforcement learning algorithm to train a Neural Network to control a four-legged robot in simulation. Reinforcement learning in general can learn complex behavior policies from simple state-reward tuples datasets and PPO in particular has proved its effectiveness in solving complex tasks with continuous states and actions. Moreover, since it is model-free, it is general and can adapt to changes in the environment or in the robot itself. The virtual environment used to train the agent was modeled using our physics engine Project Chrono. Chrono can handle non smooth dynamics simulation allowing us to introduce stiff leg-ground contacts and using its Python interface Pychrono it can be interfaced with the Machine Leaning framework TensorFlow with ease. We trained the Neural Network until it learned to control the motor torques, then various policy Neural Network input state choices have been compared.
2020
978-3-030-23131-6
978-3-030-23132-3
Training a Four Legged Robot via Deep Reinforcement Learning and Multibody Simulation / Benatti, S.; Tasora, A.; Mangoni, D.. - 53:(2020), pp. 391-398. (Intervento presentato al convegno ECCOMAS Multibody Dynamics 2019 tenutosi a Duisburg, Germany) [10.1007/978-3-030-23132-3_47].
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11381/2863509
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 1
  • ???jsp.display-item.citation.isi??? ND
social impact