A critical challenge in neurosymbolic approaches is to handle the symbol grounding problem without direct supervision. That is mapping high-dimensional raw data into an interpretation over a finite set of abstract concepts with a known meaning, without using labels. In this work, we ground symbols into sequences of images by exploiting symbolic logical knowledge in the form of Linear Temporal Logic over finite traces (LTLf) formulas, and sequence-level labels expressing if a sequence of images is compliant or not with the given formula. Our approach is based on translating the LTLf formula into an equivalent deterministic finite automaton (DFA) and interpreting the latter in fuzzy logic. Experiments show that our system outperforms recurrent neural networks in sequence classification and can reach high image classification accuracy without being trained with any single-image label.

Grounding LTLf specifications in images / Umili, Elena; Capobianco, Roberto; DE GIACOMO, Giuseppe. - (2022), pp. 45-63. (Intervento presentato al convegno 16th International Workshop on Neural-Symbolic Learning and Reasoning tenutosi a Cumberland Lodge, Windsor Great Park, United Kingdom).

Grounding LTLf specifications in images

Elena Umili
;
Roberto Capobianco;Giuseppe De Giacomo
2022

Abstract

A critical challenge in neurosymbolic approaches is to handle the symbol grounding problem without direct supervision. That is mapping high-dimensional raw data into an interpretation over a finite set of abstract concepts with a known meaning, without using labels. In this work, we ground symbols into sequences of images by exploiting symbolic logical knowledge in the form of Linear Temporal Logic over finite traces (LTLf) formulas, and sequence-level labels expressing if a sequence of images is compliant or not with the given formula. Our approach is based on translating the LTLf formula into an equivalent deterministic finite automaton (DFA) and interpreting the latter in fuzzy logic. Experiments show that our system outperforms recurrent neural networks in sequence classification and can reach high image classification accuracy without being trained with any single-image label.
2022
16th International Workshop on Neural-Symbolic Learning and Reasoning
symbol grounding; ltlf; neurosymbolic;
04 Pubblicazione in atti di convegno::04b Atto di convegno in volume
Grounding LTLf specifications in images / Umili, Elena; Capobianco, Roberto; DE GIACOMO, Giuseppe. - (2022), pp. 45-63. (Intervento presentato al convegno 16th International Workshop on Neural-Symbolic Learning and Reasoning tenutosi a Cumberland Lodge, Windsor Great Park, United Kingdom).
File allegati a questo prodotto
File Dimensione Formato  
Umili_Grounding-LTLf_2022.pdf

accesso aperto

Tipologia: Versione editoriale (versione pubblicata con il layout dell'editore)
Licenza: Creative commons
Dimensione 5.03 MB
Formato Adobe PDF
5.03 MB Adobe PDF

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11573/1657086
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 0
  • ???jsp.display-item.citation.isi??? ND
social impact