The application of machine learning to recognize hand movements from surface electromyography has led to promising academic results. Unfortunately, it has proven difficult to translate these results in better control methods for the end-users of upper-limb prostheses. Recent studies have pointed out that common offline performance metrics, such as classification accuracy, are not correlated with real controllability of the prosthesis. In this paper, we investigate the cause that learned models start to fail when applied outside the constrained laboratory setting. We performed several analyses at the hand of a dedicated data acquisition composed of a typical academic training session in the first phase and a set of activities of daily living in a home setting afterwards. Our analysis confirms that a model trained in the former setting performs poorly when applied in a home environment. The cause for this degradation is that the distribution of myoelectric data changes between both settings, thus violating the typical assumption in statistical learning theory that train and test data come from the same distribution. This problem persists even when adding data acquired in some home activities to classify others. Our result not only confirms the limited importance of offline performance metrics for real prosthesis usability, but also highlights the difficulties machine learning based approaches will need to overcome to become practically relevant.
The Difficulty of Recognizing Grasps from Semg During Activities of Daily Living / Gregori, Valentina; Caputo, Barbara; Gijsberts, Arjan. - (2018), pp. 583-588. (Intervento presentato al convegno 2018 7th IEEE International Conference on Biomedical Robotics and Biomechatronics (Biorob) tenutosi a Enschede; Netherlands) [10.1109/BIOROB.2018.8487966].
The Difficulty of Recognizing Grasps from Semg During Activities of Daily Living
Valentina Gregori
;Barbara Caputo
;Arjan Gijsberts
2018
Abstract
The application of machine learning to recognize hand movements from surface electromyography has led to promising academic results. Unfortunately, it has proven difficult to translate these results in better control methods for the end-users of upper-limb prostheses. Recent studies have pointed out that common offline performance metrics, such as classification accuracy, are not correlated with real controllability of the prosthesis. In this paper, we investigate the cause that learned models start to fail when applied outside the constrained laboratory setting. We performed several analyses at the hand of a dedicated data acquisition composed of a typical academic training session in the first phase and a set of activities of daily living in a home setting afterwards. Our analysis confirms that a model trained in the former setting performs poorly when applied in a home environment. The cause for this degradation is that the distribution of myoelectric data changes between both settings, thus violating the typical assumption in statistical learning theory that train and test data come from the same distribution. This problem persists even when adding data acquired in some home activities to classify others. Our result not only confirms the limited importance of offline performance metrics for real prosthesis usability, but also highlights the difficulties machine learning based approaches will need to overcome to become practically relevant.File | Dimensione | Formato | |
---|---|---|---|
Gregori_The-Difficulty_2018.pdf
solo gestori archivio
Tipologia:
Versione editoriale (versione pubblicata con il layout dell'editore)
Licenza:
Tutti i diritti riservati (All rights reserved)
Dimensione
1.11 MB
Formato
Adobe PDF
|
1.11 MB | Adobe PDF | Contatta l'autore |
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.