The efficient control of our body and success- ful interaction with the environment are possible through the integration of multisensory information. Brain–computer interface (BCI) may allow people with sensorimotor dis- orders to actively interact in the world. In this study, visual information was paired with auditory feedback to improve the BCI control of a humanoid surrogate. Healthy and spinal cord injured (SCI) people were asked to embody a humanoid robot and complete a pick-and- place task by means of a visual evoked potentials BCI system. Participants observed the remote environment from the robot’s perspective through a head mounted display. Human-footsteps and computer-beep sounds were used as synchronous/asynchronous auditory feedback. Healthy participants achieved better placing accuracy when lis- tening to human footstep sounds relative to a computer- generated sound. SCI people demonstrated more difficulty in steering the robot during asynchronous auditory feed- back conditions. Importantly, subjective reports highlighted that the BCI mask overlaying the display did not limit the observation of the scenario and the feeling of being in control of the robot. Overall, the data seem to suggest that sensorimotor-related information may improve the control of external devices. Further studies are required to under- stand how the contribution of residual sensory channels could improve the reliability of BCI systems.

The role of audio-visual feedback in a thought-based control of a humanoid robot: a BCI study in healthy and spinal cord injured people / Tidoni, Emmanuele; Gergondet, Pierre; Fusco, Gabriele; Kheddar, Abderrahmane; Aglioti, Salvatore Maria. - In: IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING. - ISSN 1534-4320. - ELETTRONICO. - 25:6(2017), pp. 772-781. [10.1109/TNSRE.2016.2597863]

The role of audio-visual feedback in a thought-based control of a humanoid robot: a BCI study in healthy and spinal cord injured people

TIDONI, EMMANUELE
;
FUSCO, GABRIELE;AGLIOTI, Salvatore Maria
2017

Abstract

The efficient control of our body and success- ful interaction with the environment are possible through the integration of multisensory information. Brain–computer interface (BCI) may allow people with sensorimotor dis- orders to actively interact in the world. In this study, visual information was paired with auditory feedback to improve the BCI control of a humanoid surrogate. Healthy and spinal cord injured (SCI) people were asked to embody a humanoid robot and complete a pick-and- place task by means of a visual evoked potentials BCI system. Participants observed the remote environment from the robot’s perspective through a head mounted display. Human-footsteps and computer-beep sounds were used as synchronous/asynchronous auditory feedback. Healthy participants achieved better placing accuracy when lis- tening to human footstep sounds relative to a computer- generated sound. SCI people demonstrated more difficulty in steering the robot during asynchronous auditory feed- back conditions. Importantly, subjective reports highlighted that the BCI mask overlaying the display did not limit the observation of the scenario and the feeling of being in control of the robot. Overall, the data seem to suggest that sensorimotor-related information may improve the control of external devices. Further studies are required to under- stand how the contribution of residual sensory channels could improve the reliability of BCI systems.
2017
body ownership; brain–computer interface (BCI); humanoid; spinal cord injury; steady state visually evoked potentials (SSVEP); teleoperation
01 Pubblicazione su rivista::01a Articolo in rivista
The role of audio-visual feedback in a thought-based control of a humanoid robot: a BCI study in healthy and spinal cord injured people / Tidoni, Emmanuele; Gergondet, Pierre; Fusco, Gabriele; Kheddar, Abderrahmane; Aglioti, Salvatore Maria. - In: IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING. - ISSN 1534-4320. - ELETTRONICO. - 25:6(2017), pp. 772-781. [10.1109/TNSRE.2016.2597863]
File allegati a questo prodotto
File Dimensione Formato  
Tidoni_Audio-visual-feedback_2017.pdf

solo gestori archivio

Tipologia: Versione editoriale (versione pubblicata con il layout dell'editore)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 2.42 MB
Formato Adobe PDF
2.42 MB Adobe PDF   Contatta l'autore

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11573/877840
Citazioni
  • ???jsp.display-item.citation.pmc??? 5
  • Scopus 29
  • ???jsp.display-item.citation.isi??? 25
social impact