Unsupervised dynamical learning in gated Recurrent Neural Networks

Both animals and humans demonstrate the remarkable ability to rapidly adapt their behavior, suggesting a dynamic capability within the brain to reconfigure its internal dynamics in response to varying contexts. Previous work on this topic has showcased this ability in recurrent neural networks by providing, at training time, the correct target context as a supervised signal. In this study, we propose a novel framework inspired by predictive coding theory. Here, a network is instructed to reproduce a set of different sequences, jointly learning to self-organize a latent state-space for the different contexts. The goal is achieved by refor- mulating the readout mechanism within an echo state network as a latent variable model that dynamically adapts its response in order to minimize its free-energy