We study bi-directional associative neural networks that, exposed to noisy examples of an extensive number of random archetypes, learn the latter (with or without the presence of a teacher) when the supplied information is enough: in this setting, learning is heteroassociative – involving couples of patterns – and it is achieved by reverberating the information depicted from the examples through the layers of the network. By adapting Guerra’s interpolation technique, we provide a full statistical mechanical picture of supervised and unsupervised learning processes (at the replica symmetric level of description) obtaining analytically phase diagrams, thresholds for learning, a picture of the ground-state in plain agreement with Monte Carlo simulations and signal-to-noise outcomes. In the large dataset limit, the Kosko storage prescription as well as its statistical mechanical picture provided by Kurchan, Peliti, and Saber in the eighties is fully recovered. Computational advantages in dealing with information reverberation, rather than storage, are discussed for natural test cases. In particular, we show how this network admits an integral representation in terms of two coupled restricted Boltzmann machines, whose hidden layers are entirely built of by grand-mother neurons, to prove that by coupling solely these grand-mother neurons we can correlate the patterns they are related to: it is thus possible to recover Pavlov’s Classical Conditioning by adding just one synapse among the correct grand-mother neurons (hence saving an extensive number of these links for further information storage w.r.t. the classical auto-associative setting). Likewise, it is enough to equip the BAM’s synaptic coupling with its transpose matrix (and alternate among these two, e.g., by introducing a clock) to let these networks be able count sequences of patterns (still preserving Detailed Balance, at difference w.r.t. the auto-associative case).

Statistical mechanics of learning via reverberation in bidirectional associative memories / Salomone Centonze, Martino; Kanter, Ido; Barra, Adriano. - In: PHYSICA. A. - ISSN 0378-4371. - 637:1(2024), p. 129512. [10.1016/j.physa.2024.129512]

Statistical mechanics of learning via reverberation in bidirectional associative memories

Adriano Barra
2024

Abstract

We study bi-directional associative neural networks that, exposed to noisy examples of an extensive number of random archetypes, learn the latter (with or without the presence of a teacher) when the supplied information is enough: in this setting, learning is heteroassociative – involving couples of patterns – and it is achieved by reverberating the information depicted from the examples through the layers of the network. By adapting Guerra’s interpolation technique, we provide a full statistical mechanical picture of supervised and unsupervised learning processes (at the replica symmetric level of description) obtaining analytically phase diagrams, thresholds for learning, a picture of the ground-state in plain agreement with Monte Carlo simulations and signal-to-noise outcomes. In the large dataset limit, the Kosko storage prescription as well as its statistical mechanical picture provided by Kurchan, Peliti, and Saber in the eighties is fully recovered. Computational advantages in dealing with information reverberation, rather than storage, are discussed for natural test cases. In particular, we show how this network admits an integral representation in terms of two coupled restricted Boltzmann machines, whose hidden layers are entirely built of by grand-mother neurons, to prove that by coupling solely these grand-mother neurons we can correlate the patterns they are related to: it is thus possible to recover Pavlov’s Classical Conditioning by adding just one synapse among the correct grand-mother neurons (hence saving an extensive number of these links for further information storage w.r.t. the classical auto-associative setting). Likewise, it is enough to equip the BAM’s synaptic coupling with its transpose matrix (and alternate among these two, e.g., by introducing a clock) to let these networks be able count sequences of patterns (still preserving Detailed Balance, at difference w.r.t. the auto-associative case).
2024
Statistical mechanics; Machine learning; Artificial intelligence; Explainable AI; Associative memories; Complex systems
01 Pubblicazione su rivista::01a Articolo in rivista
Statistical mechanics of learning via reverberation in bidirectional associative memories / Salomone Centonze, Martino; Kanter, Ido; Barra, Adriano. - In: PHYSICA. A. - ISSN 0378-4371. - 637:1(2024), p. 129512. [10.1016/j.physa.2024.129512]
File allegati a questo prodotto
File Dimensione Formato  
Centonze_Statistical_2024 .pdf

solo gestori archivio

Tipologia: Versione editoriale (versione pubblicata con il layout dell'editore)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 3.37 MB
Formato Adobe PDF
3.37 MB Adobe PDF   Contatta l'autore

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11573/1707710
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 0
  • ???jsp.display-item.citation.isi??? ND
social impact