Agent programming in complex, partially observable and stochastic domains usually requires a great deal of understanding of both the domain and the task, in order to provide the agent with the knowledge necessary to act effectively. While symbolic methods allow the designer to specify declarative knowledge about the domain, the resulting plan can be brittle since it is difficult to supply a symbolic model that is accurate enough to foresee all possible events in complex environments, especially in the case of partial observability. Reinforcement Learning (RL) techniques, on the other hand, can learn a policy and make use of a learned model, but it is difficult to reduce and shape the scope of the learning algorithm by exploiting a priori information. We propose a methodology for writing complex agent programs that can be effectively improved through experience. We show how to derive a stochastic process from a partial specification of the plan, so that the latter's perfomance can be improved solving a RL problem much smaller than classical RL formulations. Finally, we demonstrate our approach in the context of Keepaway Soccer, a common RL benchmark based on a RoboCup Soccer 2D simulator. Copyright © 2010, International Foundation for Autonomous Agents and Multiagent Systems (www.ifaamas.org). All rights reserved.

Improving the performance of complex agent plans through reinforcement learning / Leonetti, Matteo; Iocchi, Luca. - 2:(2010), pp. 723-730. (Intervento presentato al convegno 9th International Joint Conference on Autonomous Agents and Multiagent Systems 2010, AAMAS 2010 tenutosi a Toronto; Canada nel 10 May 2010).

Improving the performance of complex agent plans through reinforcement learning

LEONETTI, MATTEO;IOCCHI, Luca
2010

Abstract

Agent programming in complex, partially observable and stochastic domains usually requires a great deal of understanding of both the domain and the task, in order to provide the agent with the knowledge necessary to act effectively. While symbolic methods allow the designer to specify declarative knowledge about the domain, the resulting plan can be brittle since it is difficult to supply a symbolic model that is accurate enough to foresee all possible events in complex environments, especially in the case of partial observability. Reinforcement Learning (RL) techniques, on the other hand, can learn a policy and make use of a learned model, but it is difficult to reduce and shape the scope of the learning algorithm by exploiting a priori information. We propose a methodology for writing complex agent programs that can be effectively improved through experience. We show how to derive a stochastic process from a partial specification of the plan, so that the latter's perfomance can be improved solving a RL problem much smaller than classical RL formulations. Finally, we demonstrate our approach in the context of Keepaway Soccer, a common RL benchmark based on a RoboCup Soccer 2D simulator. Copyright © 2010, International Foundation for Autonomous Agents and Multiagent Systems (www.ifaamas.org). All rights reserved.
2010
9th International Joint Conference on Autonomous Agents and Multiagent Systems 2010, AAMAS 2010
learning (single and multi-agent); formal models (including robot plans; ant robots)
04 Pubblicazione in atti di convegno::04b Atto di convegno in volume
Improving the performance of complex agent plans through reinforcement learning / Leonetti, Matteo; Iocchi, Luca. - 2:(2010), pp. 723-730. (Intervento presentato al convegno 9th International Joint Conference on Autonomous Agents and Multiagent Systems 2010, AAMAS 2010 tenutosi a Toronto; Canada nel 10 May 2010).
File allegati a questo prodotto
File Dimensione Formato  
VE_2010_11573-367685.pdf

solo gestori archivio

Tipologia: Versione editoriale (versione pubblicata con il layout dell'editore)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 192.37 kB
Formato Adobe PDF
192.37 kB Adobe PDF   Contatta l'autore

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11573/367685
 Attenzione

Attenzione! I dati visualizzati non sono stati sottoposti a validazione da parte dell'ateneo

Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 5
  • ???jsp.display-item.citation.isi??? ND
social impact