This paper investigates the use of machine learning techniques for real-time optimal spacecraft guidance during terminal rendezvous maneuvers, in presence of both operational constraints and a visibility cone path constraint. Realistic stochastic effects that could lead to off-nominal conditions, such as an inaccurate knowledge of the initial spacecraft state and the presence of random in-flight disturbances, are also accounted for. The performance of two well-studied deep learning methods for control problems, Behavioral Cloning (BC) and Reinforcement Learning (RL), are investigated on a sample linear multi-impulsive rendezvous mission. To this aim, a Multi-Layer Perceptron network, with custom architecture, is designed to map any observation of the actual spacecraft state, defined by its relative position and velocity, to the propellant-optimal control action, which corresponds to a bounded-magnitude impulsive velocity variation. In the BC approach, the deep neural network is trained by supervised learning on a set of optimal trajectories, generated by routinely solving the deterministic optimal control problem via convex optimization algorithms, starting from scattered initial conditions. Conversely, in the RL approach, a state-of-the-art actor-critic algorithm, Proximal Policy Optimization (PPO), is used for training the network through repeated interactions with the stochastic environment. Eventually, the robustness and propellant-efficiency of the obtained closed-loop control policies are assessed and compared by means of a thorough Monte Carlo analysis, carried out by considering different test cases with increasing levels of perturbations.

Machine learning techniques for autonomous spacecraft guidance during proximity operation / Federici, Lorenzo; Benedikter, Boris; Zavoli, Alessandro. - (2021), pp. 1-18. (Intervento presentato al convegno AIAA Scitech 2021 Forum tenutosi a Virtual) [10.2514/6.2021-0668].

Machine learning techniques for autonomous spacecraft guidance during proximity operation

Lorenzo Federici;Boris Benedikter;Alessandro Zavoli
2021

Abstract

This paper investigates the use of machine learning techniques for real-time optimal spacecraft guidance during terminal rendezvous maneuvers, in presence of both operational constraints and a visibility cone path constraint. Realistic stochastic effects that could lead to off-nominal conditions, such as an inaccurate knowledge of the initial spacecraft state and the presence of random in-flight disturbances, are also accounted for. The performance of two well-studied deep learning methods for control problems, Behavioral Cloning (BC) and Reinforcement Learning (RL), are investigated on a sample linear multi-impulsive rendezvous mission. To this aim, a Multi-Layer Perceptron network, with custom architecture, is designed to map any observation of the actual spacecraft state, defined by its relative position and velocity, to the propellant-optimal control action, which corresponds to a bounded-magnitude impulsive velocity variation. In the BC approach, the deep neural network is trained by supervised learning on a set of optimal trajectories, generated by routinely solving the deterministic optimal control problem via convex optimization algorithms, starting from scattered initial conditions. Conversely, in the RL approach, a state-of-the-art actor-critic algorithm, Proximal Policy Optimization (PPO), is used for training the network through repeated interactions with the stochastic environment. Eventually, the robustness and propellant-efficiency of the obtained closed-loop control policies are assessed and compared by means of a thorough Monte Carlo analysis, carried out by considering different test cases with increasing levels of perturbations.
2021
AIAA Scitech 2021 Forum
reinforcement learning; spacecraft; autonomous; guidance
04 Pubblicazione in atti di convegno::04b Atto di convegno in volume
Machine learning techniques for autonomous spacecraft guidance during proximity operation / Federici, Lorenzo; Benedikter, Boris; Zavoli, Alessandro. - (2021), pp. 1-18. (Intervento presentato al convegno AIAA Scitech 2021 Forum tenutosi a Virtual) [10.2514/6.2021-0668].
File allegati a questo prodotto
File Dimensione Formato  
Federici_Machine_learning_2021.pdf

solo gestori archivio

Note: https://arc.aiaa.org/doi/10.2514/6.2021-0668
Tipologia: Versione editoriale (versione pubblicata con il layout dell'editore)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 2.31 MB
Formato Adobe PDF
2.31 MB Adobe PDF   Contatta l'autore

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11573/1482213
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 17
  • ???jsp.display-item.citation.isi??? ND
social impact