SINDy-Former: Data-driven identification of sparse dynamical models via cross-attention

Identifying compact, interpretable models for nonlinear dynamical systems solely from the analysis of noisy measurements is an open challenge in modern system identification. Symbolic regression methods, such as the well-known Sparse Identification of Nonlinear Dynamics (SINDy) algorithm, try to identify differential equations, but their reliance on numerical differentiation of the state trajectories makes them particularly sensitive to measurement noise. In this context, we propose SINDy-Former, a novel neural network architecture that combines the concept of a library of candidate functions, inspired by SINDy, with the multi-head cross-attention mechanism that characterises transformers. Our solution is capable of learning a sparse coefficient matrix directly from the raw trajectory data, avoiding the need for numerical differentiation thanks to a custom training loss function. We validate our solution on three nonlinear dynamical systems, including the chaotic Halvorsen attractor and the Lorenz-96 up to dimension 15, considering noise-corrupted measurements and samplescarce scenarios. Overall, SINDy-Former bridges the gap between high-capacity neural networks and transparent dynamics modelling, avoiding black-box limitations in favour of a fully interpretable model, hence providing a robust framework for data-driven discovery in complex dynamical systems.