The application of machine learning to recognize hand movements from surface electromyography has led to promising academic results. Unfortunately, it has proven difficult to translate these results in better control methods for the end-users of upper-limb prostheses. Recent studies have pointed out that common offline performance metrics, such as classification accuracy, are not correlated with real controllability of the prosthesis. In this paper, we investigate the cause that learned models start to fail when applied outside the constrained laboratory setting. We performed several analyses at the hand of a dedicated data acquisition composed of a typical academic training session in the first phase and a set of activities of daily living in a home setting afterwards. Our analysis confirms that a model trained in the former setting performs poorly when applied in a home environment. The cause for this degradation is that the distribution of myoelectric data changes between both settings, thus violating the typical assumption in statistical learning theory that train and test data come from the same distribution. This problem persists even when adding data acquired in some home activities to classify others. Our result not only confirms the limited importance of offline performance metrics for real prosthesis usability, but also highlights the difficulties machine learning based approaches will need to overcome to become practically relevant.

The Difficulty of Recognizing Grasps from Semg During Activities of Daily Living / Gregori, Valentina; Caputo, Barbara; Gijsberts, Arjan. - (2018), pp. 583-588. (Intervento presentato al convegno 2018 7th IEEE International Conference on Biomedical Robotics and Biomechatronics (Biorob) tenutosi a Enschede; Netherlands) [10.1109/BIOROB.2018.8487966].

The Difficulty of Recognizing Grasps from Semg During Activities of Daily Living

Valentina Gregori
;
Barbara Caputo
;
Arjan Gijsberts
2018

Abstract

The application of machine learning to recognize hand movements from surface electromyography has led to promising academic results. Unfortunately, it has proven difficult to translate these results in better control methods for the end-users of upper-limb prostheses. Recent studies have pointed out that common offline performance metrics, such as classification accuracy, are not correlated with real controllability of the prosthesis. In this paper, we investigate the cause that learned models start to fail when applied outside the constrained laboratory setting. We performed several analyses at the hand of a dedicated data acquisition composed of a typical academic training session in the first phase and a set of activities of daily living in a home setting afterwards. Our analysis confirms that a model trained in the former setting performs poorly when applied in a home environment. The cause for this degradation is that the distribution of myoelectric data changes between both settings, thus violating the typical assumption in statistical learning theory that train and test data come from the same distribution. This problem persists even when adding data acquired in some home activities to classify others. Our result not only confirms the limited importance of offline performance metrics for real prosthesis usability, but also highlights the difficulties machine learning based approaches will need to overcome to become practically relevant.
2018
2018 7th IEEE International Conference on Biomedical Robotics and Biomechatronics (Biorob)
Prosthetics; Artificial intelligence; Data acquisition; Prosthetics; Robotics
04 Pubblicazione in atti di convegno::04b Atto di convegno in volume
The Difficulty of Recognizing Grasps from Semg During Activities of Daily Living / Gregori, Valentina; Caputo, Barbara; Gijsberts, Arjan. - (2018), pp. 583-588. (Intervento presentato al convegno 2018 7th IEEE International Conference on Biomedical Robotics and Biomechatronics (Biorob) tenutosi a Enschede; Netherlands) [10.1109/BIOROB.2018.8487966].
File allegati a questo prodotto
File Dimensione Formato  
Gregori_The-Difficulty_2018.pdf

solo gestori archivio

Tipologia: Versione editoriale (versione pubblicata con il layout dell'editore)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 1.11 MB
Formato Adobe PDF
1.11 MB Adobe PDF   Contatta l'autore

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11573/1321407
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 0
  • ???jsp.display-item.citation.isi??? ND
social impact