A Deep Reinforcement Learning Approach to Autonomous Spacecraft Docking

A great deal of research interest is currently granted to enhance the autonomy of space systems, with particular attention to future on-orbit servicing and docking operations. In this scenario, modern machine learning algorithms are a key asset to the development of such activities. This paper aims at giving a contribution in this field by implementing a Deep Reinforcement Learning (DRL) Actor/Critic approach as a feedback control law for a three-degrees-of-freedom autonomous docking manoeuvre. In detail, the agent's policy is estimated to map a set of generally available observations (i.e. spacecraft attitude, position and corresponding rates) to a group of actions (represented by the commands exerted on the chaser spacecraft) to maximize a given reward signal. The policy is learned to successfully carry out the manoeuvre while both preventing collisions and respecting constraints in terms of docking conditions, without relying on pre-programmed reference controllers. To this purpose, the DRL framework is developed in Matlab/Simulink environment by coupling three different Matlab tools, namely Simscape Multibody to simulate the spacecraft dynamics, the Reinforcement Learning Toolbox to set-up the learning environment and the Deep Learning Toolbox to design the DLR policy Neural Networks. Finally, simulations are carried out to verify the efficacy of the proposed solution, aiming at offering ground for further developments.