LearnPNP: A tool for learning agent behaviors

High-level programming of robotic agents requires the use of a representation formalism able to cope with several sources of complexity (e.g. parallel execution, partial observability, exogenous events, etc.) and the ability of the designer to model the domain in a precise way. Reinforcement Learning has proved promising in improving the performance, adaptability and robustness of plans in under-specified domains, although it does not scale well with the complexity of common robotic applications. In this paper we propose to combine an extremely expressive plan representation formalism (Petri Net Plans), with Reinforcement Learning over a stochastic process derived directly from such a plan. The derived process has a significantly reduced search space and thus the framework scales well with the complexity of the domain and allows for actually improving the performance of complex behaviors from experience. To prove the effectiveness of the system, we show how to model and learn the behavior of the robotic agents in the context of Keepaway Soccer (a widely accepted benchmark for RL) and the RoboCup Standard Platform League. © 2011 Springer-Verlag.