In this work a novel approach to the training of recurrent neural nets is presented. the algorithm exploits the separability of each neuron into its linear and nonlinear part. Each iteration of the learning consists of two steps: first the descent of the error functional in the space of the linear outputs of the neurons is performed (descent in the neuron space); then the weights are updated by solving a linear system with a Recursive Least squares technique. The main properties of the new approach are high speed of convergence, favorable numerical conditioning and robustness. The numerical stability is assured by the use of robust LS linear system solvers, operating directly on the data.
Fast training of Recurrent Neural Networks by the Recursive Least Squares Method / Parisi, Raffaele; DI CLAUDIO, Elio; Rapagnetta, A.; Orlandi, Gianni. - STAMPA. - (1997), pp. 338-343. (Intervento presentato al convegno VIII Italian Workshop on Neural Nets - WIRN Vietri 96 tenutosi a Vietri sul mare (SA), Italy nel 23-25 Maggio 1996).
Fast training of Recurrent Neural Networks by the Recursive Least Squares Method
PARISI, Raffaele;DI CLAUDIO, Elio;ORLANDI, Gianni
1997
Abstract
In this work a novel approach to the training of recurrent neural nets is presented. the algorithm exploits the separability of each neuron into its linear and nonlinear part. Each iteration of the learning consists of two steps: first the descent of the error functional in the space of the linear outputs of the neurons is performed (descent in the neuron space); then the weights are updated by solving a linear system with a Recursive Least squares technique. The main properties of the new approach are high speed of convergence, favorable numerical conditioning and robustness. The numerical stability is assured by the use of robust LS linear system solvers, operating directly on the data.I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.