Testing audio compression autoencoders for seismology. Moving toward foundation models

Efforts to develop foundation models (FMs) for seismic waveform analysis are beginning to advance with progress still needed for geophysics applications. FMs learn a generalized representation of the data using a self-supervised approach, thus allowing several downstream tasks to be performed in a unified framework. We test a seismic waveform autoencoder as a FM, that is, pretrained for audio data compression and transfer this to seismic data. The pre-trained audio model is fine-tuned using seismic waveforms and evaluated with a waveform reconstruction task. The compression and reconstruction encourage the model to learn a generalizable representation of the input signal. The fine-tuned model is tested for generalization using waveforms from different data distributions. The embedding signal is applied to downstream tasks for foreshock and aftershock classification, early warning ground motion estimates, and earthquake phase detection. We systematically test different fine-tuning procedures to evaluate the effectiveness of the pre-trained model encoder for producing a generalized hidden space. The results show improvement when compared to task-specific models, even when applying few-shot learning. The performance of this model highlights the strengths and limitations as a FM and provides insight for future efforts to develop FMs for seismology.