In this paper we present a novel mechanism to get explanations that allow to better understand network predictions when dealing with sequential data. Specifically, we adopt memory-based networks - Differential Neural Computers - to exploit their capability of storing data in memory and reusing it for inference. By tracking both the memory access at prediction time, and the information stored by the network at each step of the input sequence, we can retrieve the most relevant input steps associated to each prediction. We validate our approach (1) on a modified T-maze, which is a non-Markovian discrete control task evaluating an algorithm's ability to correlate events far apart in history, and (2) on the Story Cloze Test, which is a commonsense reasoning framework for evaluating story understanding that requires a system to choose the correct ending to a four-sentence story. Our results show that we are able to explain agent's decisions in (1) and to reconstruct the most relevant sentences used by the network to select the story ending in (2). Additionally, we show not only that by removing those sentences the network prediction changes, but also that the same are sufficient to reproduce the inference.

Explainable inference on sequential data via memory-tracking / LA ROSA, Biagio; Capobianco, Roberto; Nardi, Daniele. - (2021), pp. 2006-2013. (Intervento presentato al convegno International Joint Conference on Artificial Intelligence tenutosi a Yokohama; Japan).

Explainable inference on sequential data via memory-tracking

Biagio La Rosa;Roberto Capobianco
Co-primo
;
Daniele Nardi
2021

Abstract

In this paper we present a novel mechanism to get explanations that allow to better understand network predictions when dealing with sequential data. Specifically, we adopt memory-based networks - Differential Neural Computers - to exploit their capability of storing data in memory and reusing it for inference. By tracking both the memory access at prediction time, and the information stored by the network at each step of the input sequence, we can retrieve the most relevant input steps associated to each prediction. We validate our approach (1) on a modified T-maze, which is a non-Markovian discrete control task evaluating an algorithm's ability to correlate events far apart in history, and (2) on the Story Cloze Test, which is a commonsense reasoning framework for evaluating story understanding that requires a system to choose the correct ending to a four-sentence story. Our results show that we are able to explain agent's decisions in (1) and to reconstruct the most relevant sentences used by the network to select the story ending in (2). Additionally, we show not only that by removing those sentences the network prediction changes, but also that the same are sufficient to reproduce the inference.
2021
International Joint Conference on Artificial Intelligence
explainability; sequential data; machine learning; deep learning
04 Pubblicazione in atti di convegno::04b Atto di convegno in volume
Explainable inference on sequential data via memory-tracking / LA ROSA, Biagio; Capobianco, Roberto; Nardi, Daniele. - (2021), pp. 2006-2013. (Intervento presentato al convegno International Joint Conference on Artificial Intelligence tenutosi a Yokohama; Japan).
File allegati a questo prodotto
File Dimensione Formato  
LaRosa_Explainable_2021.pdf

accesso aperto

Note: https://www.ijcai.org/proceedings/2020/278
Tipologia: Versione editoriale (versione pubblicata con il layout dell'editore)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 325.19 kB
Formato Adobe PDF
325.19 kB Adobe PDF

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11573/1397019
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 7
  • ???jsp.display-item.citation.isi??? 4
social impact