In neural network's literature, Hebbian learning traditionally refers to the procedure by which the Hopfield model and its generalizations store archetypes (i.e., definite patterns that are experienced just once to form the synaptic matrix). However, the term learning in machine learning refers to the ability of the machine to extract features from the supplied dataset (e.g., made of blurred examples of these archetypes), in order to make its own representation of the unavailable archetypes. Here, given a sample of examples, we define a supervised learning protocol based on Hebb's rule and by which the Hopfield network can infer the archetypes. By an analytical inspection, we detect the correct control parameters (including size and quality of the dataset) that tune the system performance and we depict its phase diagram. We also prove that, for struc-tureless datasets, the Hopfield model equipped with this supervised learning rule is equivalent to a restricted Boltzmann machine and this suggests an optimal and interpretable training routine. Finally, this approach is generalized to structured datasets: we highlight an ultrametric-like orga-nization (reminiscent of replica-symmetry-breaking) in the analyzed datasets and, consequently, we introduce an additional broken-replica hidden layer for its (partial) disentanglement, which is shown to improve MNIST classification from 75% to 95%, and to offer a new perspective on deep architectures.

Supervised Hebbian learning / Alemanno, Francesco; Aquaro, Miriam; Kanter, Ido; Barra, Adriano; Agliari, Elena. - In: EUROPHYSICS LETTERS. - ISSN 0295-5075. - 141:1(2023), p. 11001. [10.1209/0295-5075/aca55f]

Supervised Hebbian learning

Miriam Aquaro;Adriano Barra;Elena Agliari
2023

Abstract

In neural network's literature, Hebbian learning traditionally refers to the procedure by which the Hopfield model and its generalizations store archetypes (i.e., definite patterns that are experienced just once to form the synaptic matrix). However, the term learning in machine learning refers to the ability of the machine to extract features from the supplied dataset (e.g., made of blurred examples of these archetypes), in order to make its own representation of the unavailable archetypes. Here, given a sample of examples, we define a supervised learning protocol based on Hebb's rule and by which the Hopfield network can infer the archetypes. By an analytical inspection, we detect the correct control parameters (including size and quality of the dataset) that tune the system performance and we depict its phase diagram. We also prove that, for struc-tureless datasets, the Hopfield model equipped with this supervised learning rule is equivalent to a restricted Boltzmann machine and this suggests an optimal and interpretable training routine. Finally, this approach is generalized to structured datasets: we highlight an ultrametric-like orga-nization (reminiscent of replica-symmetry-breaking) in the analyzed datasets and, consequently, we introduce an additional broken-replica hidden layer for its (partial) disentanglement, which is shown to improve MNIST classification from 75% to 95%, and to offer a new perspective on deep architectures.
2023
neural-networks; statistical-mechanics; artificial-intelligence; machine-learning; mathematical-physics
01 Pubblicazione su rivista::01a Articolo in rivista
Supervised Hebbian learning / Alemanno, Francesco; Aquaro, Miriam; Kanter, Ido; Barra, Adriano; Agliari, Elena. - In: EUROPHYSICS LETTERS. - ISSN 0295-5075. - 141:1(2023), p. 11001. [10.1209/0295-5075/aca55f]
File allegati a questo prodotto
File Dimensione Formato  
Alemanno_Supervised_2023.pdf

accesso aperto

Tipologia: Versione editoriale (versione pubblicata con il layout dell'editore)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 848.98 kB
Formato Adobe PDF
848.98 kB Adobe PDF

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11573/1671043
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 12
  • ???jsp.display-item.citation.isi??? 10
social impact