Transformer neural networks have revolutionized machine learning, excelling in text and image processing. Their self-attention mechanism captures sequence dependencies, facilitating feature extraction and avoiding gradient problems of recurrent networks. Transformers naturally implement a meta-reinforcement learning framework when used in reinforcement learning, using self-attention weights as context-dependent parameters for task inference. This paper proposes a meta-reinforcement learning algorithm based on the gated transformerXL model for autonomous spacecraft guidance during a planetary landing, by considering the presence of unmodeled dynamics, inaccurate navigation data, and control errors. The method will be compared with standard reinforcement learning via a feed-forward network to demonstrate the potential of transformers for real-time and robust spacecraft guidance in uncertain mission scenarios.
Meta-Reinforcement Learning with Transformer Networks for Space Guidance Applications / Federici, Lorenzo; Furfaro, Roberto. - (2024). (Intervento presentato al convegno AIAA SciTech Forum and Exposition, 2024 tenutosi a Orlando; Florida (USA)) [10.2514/6.2024-2061].
Meta-Reinforcement Learning with Transformer Networks for Space Guidance Applications
Federici, Lorenzo;
2024
Abstract
Transformer neural networks have revolutionized machine learning, excelling in text and image processing. Their self-attention mechanism captures sequence dependencies, facilitating feature extraction and avoiding gradient problems of recurrent networks. Transformers naturally implement a meta-reinforcement learning framework when used in reinforcement learning, using self-attention weights as context-dependent parameters for task inference. This paper proposes a meta-reinforcement learning algorithm based on the gated transformerXL model for autonomous spacecraft guidance during a planetary landing, by considering the presence of unmodeled dynamics, inaccurate navigation data, and control errors. The method will be compared with standard reinforcement learning via a feed-forward network to demonstrate the potential of transformers for real-time and robust spacecraft guidance in uncertain mission scenarios.I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.