Machine learning techniques for autonomous spacecraft guidance during proximity operation

This paper investigates the use of machine learning techniques for real-time optimal spacecraft guidance during terminal rendezvous maneuvers, in presence of both operational constraints and a visibility cone path constraint. Realistic stochastic effects that could lead to off-nominal conditions, such as an inaccurate knowledge of the initial spacecraft state and the presence of random in-flight disturbances, are also accounted for. The performance of two well-studied deep learning methods for control problems, Behavioral Cloning (BC) and Reinforcement Learning (RL), are investigated on a sample linear multi-impulsive rendezvous mission. To this aim, a Multi-Layer Perceptron network, with custom architecture, is designed to map any observation of the actual spacecraft state, defined by its relative position and velocity, to the propellant-optimal control action, which corresponds to a bounded-magnitude impulsive velocity variation. In the BC approach, the deep neural network is trained by supervised learning on a set of optimal trajectories, generated by routinely solving the deterministic optimal control problem via convex optimization algorithms, starting from scattered initial conditions. Conversely, in the RL approach, a state-of-the-art actor-critic algorithm, Proximal Policy Optimization (PPO), is used for training the network through repeated interactions with the stochastic environment. Eventually, the robustness and propellant-efficiency of the obtained closed-loop control policies are assessed and compared by means of a thorough Monte Carlo analysis, carried out by considering different test cases with increasing levels of perturbations.