Practical Implementation of “Sim-to-Real” Deep Reinforcement Learning Control for Inverted Pendulum System

Abstract

The following work describes how it was possible to use “sim-to-real” learning todesign the deep reinforcement learning controller for the inverted pendulum. The “sim-to-real” learning is the branch of transfer learning that deals with transmitting knowledge from virtual environments to real-world applications. By following the “sim-to-real” methods, the controller for the inverted pendulum has been created from experience obtained from its digital twin. After optimizing the dynamics of the virtual model using the System Identification method, it was possible to reduce the training time for the physical twin by three times compared to the controller trained without prior experience. The paper also shows the suggested design ofa deep reinforcement learning platform, a software library that aids in connecting the reinforcement learning training agent with control objects that lack a direct mechanism of communicating observations with the training agent. Throughout the research document, the reader will get familiar with all the technical steps required to achieve a successful “sim-to-real” experiment.