Block Layer decomposition applied to a watchdog controlled minibatch algorithm

In this paper, we introduce a decomposed version of the CMA Light algorithm, leveraging block decomposition over variables to enhance computational efficiency by significantly reducing computational time while maintaining satisfactory performance. Our approach dynamically selects the blocks of variables to be updated at each iteration, based on both training performance conditions and architectural importance heuristics. Numerical results demonstrate that this strategy achieves a favorable trade-off between substantially reducing the computational cost while maintaining sufficient accuracy. This makes it a suitable and robust alternative in application where high precision is not essential or computational resources are limited.