The study of human engagement has significantly grown in recent years, particularly accelerated by the interaction with a growing number of smart computing machines [1, 2, 3]. Engagement estimation has significant importance across various domains of study, including advertising, marketing, human-computer interaction, and healthcare [4, 5, 6]. In this paper, we propose a real-time application that leverages a single RGB camera to capture user behavior. Our approach implements a novel method for estimating human engagement in real-world scenarios by extracting valuable information from the combination of facial expressions and gaze direction analysis. To acquire this data, we employed fast and accurate machine learning algorithms from the external library dlib, along with custom versions of Residual Neural Networks implemented from scratch. For training our models, we used a modified version of the DAiSEE dataset, a multi-label user affective states classification dataset that collects frontal videos of 112 different people recorded in real-world scenarios. In the absence of a baseline for comparing the results obtained by our application, we conducted experiments to assess its robustness in estimating engagement levels, leading to very encouraging results.

A Machine Learning based Real-Time Application for Engagement Detection / Iacobelli, E.; Russo, S.; Napoli, C.. - 3695:(2023), pp. 75-84. (Intervento presentato al convegno 9th Scholar's Yearly Symposium of Technology, Engineering and Mathematics, SYSTEM 2023 tenutosi a Roma; Italia).

A Machine Learning based Real-Time Application for Engagement Detection

Iacobelli E.
Co-primo
;
Russo S.
Co-primo
;
Napoli C.
Ultimo
2023

Abstract

The study of human engagement has significantly grown in recent years, particularly accelerated by the interaction with a growing number of smart computing machines [1, 2, 3]. Engagement estimation has significant importance across various domains of study, including advertising, marketing, human-computer interaction, and healthcare [4, 5, 6]. In this paper, we propose a real-time application that leverages a single RGB camera to capture user behavior. Our approach implements a novel method for estimating human engagement in real-world scenarios by extracting valuable information from the combination of facial expressions and gaze direction analysis. To acquire this data, we employed fast and accurate machine learning algorithms from the external library dlib, along with custom versions of Residual Neural Networks implemented from scratch. For training our models, we used a modified version of the DAiSEE dataset, a multi-label user affective states classification dataset that collects frontal videos of 112 different people recorded in real-world scenarios. In the absence of a baseline for comparing the results obtained by our application, we conducted experiments to assess its robustness in estimating engagement levels, leading to very encouraging results.
2023
9th Scholar's Yearly Symposium of Technology, Engineering and Mathematics, SYSTEM 2023
Engagement Detection; Eye Tracking; Face Expression Recognition; Machine Learning; Residual Neural Networks
04 Pubblicazione in atti di convegno::04b Atto di convegno in volume
A Machine Learning based Real-Time Application for Engagement Detection / Iacobelli, E.; Russo, S.; Napoli, C.. - 3695:(2023), pp. 75-84. (Intervento presentato al convegno 9th Scholar's Yearly Symposium of Technology, Engineering and Mathematics, SYSTEM 2023 tenutosi a Roma; Italia).
File allegati a questo prodotto
File Dimensione Formato  
Iacobelli_A Machine_2023.pdf

accesso aperto

Note: https://ceur-ws.org/Vol-3695/p10.pdf
Tipologia: Versione editoriale (versione pubblicata con il layout dell'editore)
Licenza: Creative commons
Dimensione 1.65 MB
Formato Adobe PDF
1.65 MB Adobe PDF

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11573/1712983
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 0
  • ???jsp.display-item.citation.isi??? ND
social impact