Intermediate Layers of LLMs Align Best With the Brain by Balancing Short- and Long-Range Information

Contextual integration is fundamental to human language comprehension. Language models are a powerful tool for studying how contextual information influences brain activity. In this work, we analyze the brain alignment of three types of language models, which vary in how they integrate contextual information. Despite differences among models, we find minimal variations in their brain alignment. In line with previous research, middle layers consistently show the highest correspondence with brain activity. Interestingly, this alignment appears to strengthen with longer context inputs, pointing to improved sensitivity to extended linguistic information. To better understand how contextual integration affects brain alignment, we analyze the roles of short- and long-range context using variance partitioning. Our findings highlight a functional distinction between layers, suggesting a trade-off between retaining local detail and integrating broader context. This interplay may explain the robust alignment of middle layers with brain responses.