Inspired by a formal equivalence between the Hopfield model and restricted Boltzmann machines (RBMs), we design a Boltzmann machine, referred to as the dreaming Boltzmann machine (DBM), which achieves better performances than the standard one. The novelty in our model lies in a precise prescription for intralayer connections among hidden neurons whose strengths depend on features correlations. We analyze learning and retrieving capabilities in DBMs, both theoretically and numerically, and compare them to the RBM reference. We find that, in a supervised scenario, the former significantly outperforms the latter. Furthermore, in the unsupervised case, the DBM achieves better performances both in features extraction and representation learning, especially when the network is properly pretrained. Finally, we compare both models in simple classification tasks and find that the DBM again outperforms the RBM reference.

Outperforming RBM Feature-Extraction Capabilities by "Dreaming" Mechanism

Fachechi, Alberto;Agliari, Elena;
2022

Abstract

Inspired by a formal equivalence between the Hopfield model and restricted Boltzmann machines (RBMs), we design a Boltzmann machine, referred to as the dreaming Boltzmann machine (DBM), which achieves better performances than the standard one. The novelty in our model lies in a precise prescription for intralayer connections among hidden neurons whose strengths depend on features correlations. We analyze learning and retrieving capabilities in DBMs, both theoretically and numerically, and compare them to the RBM reference. We find that, in a supervised scenario, the former significantly outperforms the latter. Furthermore, in the unsupervised case, the DBM achieves better performances both in features extraction and representation learning, especially when the network is properly pretrained. Finally, we compare both models in simple classification tasks and find that the DBM again outperforms the RBM reference.
File allegati a questo prodotto
File Dimensione Formato  
Fachech_Outperforming RBM_2022.pdf

accesso aperto

Tipologia: Versione editoriale (versione pubblicata con il layout dell'editore)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 3.89 MB
Formato Adobe PDF
3.89 MB Adobe PDF Visualizza/Apri PDF

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11573/1658172
Citazioni
  • ???jsp.display-item.citation.pmc??? 0
  • Scopus 0
  • ???jsp.display-item.citation.isi??? 0
social impact