We introduce and study Regular Decision Processes (RDPs), a new, compact, factored model for domains with non-Markovian dynamics and rewards. In RDPs, transition and reward functions are specified using formulas in linear dynamic logic over finite traces, a language with the expressive power of regular expressions. This allows specifying complex dependence on the past using intuitive and compact formulas, and provides a model that generalizes MDPs and k-order MDPs. RDPs can also approximate POMDPs without having to postulate the existence of hidden variables, and, in principle, can be learned from observations only.
Regular decision processes: A model for non-markovian domains / Brafman, R. I.; De Giacomo, G.. - In: IJCAI. - ISSN 1045-0823. - (2019), pp. 5516-5522. (Intervento presentato al convegno 28th International Joint Conference on Artificial Intelligence, IJCAI 2019 tenutosi a Macao; China).
Regular decision processes: A model for non-markovian domains
Brafman R. I.
;De Giacomo G.
2019
Abstract
We introduce and study Regular Decision Processes (RDPs), a new, compact, factored model for domains with non-Markovian dynamics and rewards. In RDPs, transition and reward functions are specified using formulas in linear dynamic logic over finite traces, a language with the expressive power of regular expressions. This allows specifying complex dependence on the past using intuitive and compact formulas, and provides a model that generalizes MDPs and k-order MDPs. RDPs can also approximate POMDPs without having to postulate the existence of hidden variables, and, in principle, can be learned from observations only.File | Dimensione | Formato | |
---|---|---|---|
Brafman_Regular_2019.pdf
solo gestori archivio
Tipologia:
Versione editoriale (versione pubblicata con il layout dell'editore)
Licenza:
Tutti i diritti riservati (All rights reserved)
Dimensione
154.61 kB
Formato
Adobe PDF
|
154.61 kB | Adobe PDF | Contatta l'autore |
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.