We consider an LQR optimal control problem with partially unknown dynamics. We propose a new model-based online algorithm to obtain an approximation of the dynamics and the control at the same time during a single simulation. The iterative algorithm is based on a mixture of Reinforcement Learning and optimal control techniques. In particular, we use Gaussian distributions to represent model uncertainty and the probabilistic model is updated at each iteration using Bayesian regression formulas. On the other hand, the control is obtained in feedback form via a Riccati differential equation. We present some numerical tests showing that the algorithm can efficiently bring the system towards the origin.

A New Algorithm for the LQR Problem with Partially Unknown Dynamics / Pacifico, Agnese; Pesare, Andrea; Falcone, Maurizio. - 13127:(2022), pp. 322-330. (Intervento presentato al convegno 13th International Conference on Large-Scale Scientific Computations, LSSC 2021 tenutosi a Sozopol, Bulgaria) [10.1007/978-3-030-97549-4_37].

A New Algorithm for the LQR Problem with Partially Unknown Dynamics

Pacifico, Agnese
;
Pesare, Andrea;Falcone, Maurizio
2022

Abstract

We consider an LQR optimal control problem with partially unknown dynamics. We propose a new model-based online algorithm to obtain an approximation of the dynamics and the control at the same time during a single simulation. The iterative algorithm is based on a mixture of Reinforcement Learning and optimal control techniques. In particular, we use Gaussian distributions to represent model uncertainty and the probabilistic model is updated at each iteration using Bayesian regression formulas. On the other hand, the control is obtained in feedback form via a Riccati differential equation. We present some numerical tests showing that the algorithm can efficiently bring the system towards the origin.
2022
13th International Conference on Large-Scale Scientific Computations, LSSC 2021
reinforcement learning; lqr problem; numerical methods
04 Pubblicazione in atti di convegno::04b Atto di convegno in volume
A New Algorithm for the LQR Problem with Partially Unknown Dynamics / Pacifico, Agnese; Pesare, Andrea; Falcone, Maurizio. - 13127:(2022), pp. 322-330. (Intervento presentato al convegno 13th International Conference on Large-Scale Scientific Computations, LSSC 2021 tenutosi a Sozopol, Bulgaria) [10.1007/978-3-030-97549-4_37].
File allegati a questo prodotto
File Dimensione Formato  
Pacifico_Algorithm_2021.pdf

Open Access dal 19/03/2023

Tipologia: Documento in Post-print (versione successiva alla peer review e accettata per la pubblicazione)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 356.52 kB
Formato Adobe PDF
356.52 kB Adobe PDF
Pacifico_indice_Algorithm_2022.pdf

solo gestori archivio

Tipologia: Altro materiale allegato
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 156.55 kB
Formato Adobe PDF
156.55 kB Adobe PDF   Contatta l'autore
Pacifico_copertina_Algorithm_2022.pdf

solo gestori archivio

Tipologia: Altro materiale allegato
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 96.01 kB
Formato Adobe PDF
96.01 kB Adobe PDF   Contatta l'autore
Pacifico_versione_editoriale_A_New_Algorithm_2022.pdf

accesso aperto

Tipologia: Versione editoriale (versione pubblicata con il layout dell'editore)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 258.83 kB
Formato Adobe PDF
258.83 kB Adobe PDF

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11573/1623273
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 2
  • ???jsp.display-item.citation.isi??? ND
social impact