Essentially cyclic asynchronous nonconvex large-scale optimization

We propose a novel parallel essentially cyclic asynchronous algorithm for the minimization of the sum of a smooth (nonconvex) function and a convex (nonsmooth) regularizer. The framework hinges on Successive Convex Approximation (SCA) techniques and on a new global model that describes many asynchronous environments in a more faithful and exhaustive way with respect to state-of-the-art models. A key feature of the model is the update of (block) variables according to the essential cyclic rule: an integer B exists such that, every B iterations, all (block) variables are updated at least once. The algorithm is theoretically guaranteed to achieve a sublinear convergence rate and near linear speedup with respect to the number of cores. Asymptotic convergence to stationary solutions is also proved. Numerical results show that our scheme compares favorably to existing asynchronous methods.