Block-recursive least squares technique for training multilayer perceptrons

A novel learning technique is described as a faster and more reliable alternative to the classical back-propagation method. The approach is based on the application of Least Squares criterion to a linearized system at each step of the learning procedure. The squared error at the output of each layer immediately before the non linearity is minimized over the entire training set by a Block recursive Least Squares algorithm. The optimal weights (in the sense of minimal 2-norm of the error) are computed for each layer using the QR decomposition. The high performance of the new algorithm has been verified in several experimental trials, yielding considerable improvements from the point of view of both the accuracy and the speed of convergence.