Understanding the structure of interaction processes helps us to improve information-seeking dialogue systems. Analyzing an interaction process boils down to discovering patterns in sequences of alternating utterances exchanged between a user and an agent. Process mining techniques have been successfully applied to analyze structured event logs, discovering the underlying process models or evaluating whether the observed behavior is in conformance with the known process. In this paper, we apply process mining techniques to discover patterns in conversational transcripts and extract a new model of information-seeking dialogues, QRFA, for Query, Request, Feedback, Answer. Our results are grounded in an empirical evaluation across multiple conversational datasets from different domains, which was never attempted before. We show that the QRFA model better reflects conversation flows observed in real information-seeking conversations than models proposed previously. Moreover, QRFA allows us to identify malfunctioning in dialogue system transcripts as deviations from the expected conversation flow described by the model via conformance analysis.

QRFA: A data-driven model of information-seeking dialogues / Vakulenko, S.; Revoredo, K.; Di Ciccio, C.; de Rijke, M.. - 11437:(2019), pp. 541-557. (Intervento presentato al convegno 41st European Conference on Information Retrieval, ECIR 2019 tenutosi a Cologne; Germany) [10.1007/978-3-030-15712-8_35].

QRFA: A data-driven model of information-seeking dialogues

Di Ciccio C.;
2019

Abstract

Understanding the structure of interaction processes helps us to improve information-seeking dialogue systems. Analyzing an interaction process boils down to discovering patterns in sequences of alternating utterances exchanged between a user and an agent. Process mining techniques have been successfully applied to analyze structured event logs, discovering the underlying process models or evaluating whether the observed behavior is in conformance with the known process. In this paper, we apply process mining techniques to discover patterns in conversational transcripts and extract a new model of information-seeking dialogues, QRFA, for Query, Request, Feedback, Answer. Our results are grounded in an empirical evaluation across multiple conversational datasets from different domains, which was never attempted before. We show that the QRFA model better reflects conversation flows observed in real information-seeking conversations than models proposed previously. Moreover, QRFA allows us to identify malfunctioning in dialogue system transcripts as deviations from the expected conversation flow described by the model via conformance analysis.
2019
41st European Conference on Information Retrieval, ECIR 2019
conversational search; log analysis; process mining
04 Pubblicazione in atti di convegno::04b Atto di convegno in volume
QRFA: A data-driven model of information-seeking dialogues / Vakulenko, S.; Revoredo, K.; Di Ciccio, C.; de Rijke, M.. - 11437:(2019), pp. 541-557. (Intervento presentato al convegno 41st European Conference on Information Retrieval, ECIR 2019 tenutosi a Cologne; Germany) [10.1007/978-3-030-15712-8_35].
File allegati a questo prodotto
File Dimensione Formato  
Vakulenko_QRFA_2019.pdf

solo gestori archivio

Tipologia: Versione editoriale (versione pubblicata con il layout dell'editore)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 1.24 MB
Formato Adobe PDF
1.24 MB Adobe PDF   Contatta l'autore
Vakulenko_postprint_QRFA_2019.pdf

accesso aperto

Tipologia: Documento in Post-print (versione successiva alla peer review e accettata per la pubblicazione)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 977.55 kB
Formato Adobe PDF
977.55 kB Adobe PDF

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11573/1362050
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 27
  • ???jsp.display-item.citation.isi??? ND
social impact