Arm-Hand Behaviours Modelling: From Attention to Imitation

We present a new and original method for modelling arm-hand actions, learning and recognition. We use an incremental approach to separate the arm-hand action recognition problem into three levels. The lower level exploits bottom-up attention to select the region of interest, and attention is specifically tuned towards human motion. The middle level serves to classify action primitives exploiting motion features as descriptors. Each of the primitives is modelled by a Mixture of Gaussian, and it is recognised by a complete, real time and robust recognition system. The higher level system combines sequences of primitives using deterministic finite automata. The contribution of the paper is a compositional based model for arm-hand behaviours allowing a robot to learn new actions in a one time shot demonstration of the action execution.