Markerless Pose Estimation of DeepLabCut for Shoulder Motion Assessment in Patients with Cervical Spinal Cord Injury

Purpose: Residual motion of upper limbs in individuals with cervical spinal cord injury (CSCI) is a key-element in achieving functional independence and performing activities of daily living. Objective assessment of shoulder joint range of motion (ROM) is crucial to monitor CSCI patients’ progress. Advancements in technological devices allowed to validate wearable Inertial Measurement Unit sensors (IMU) to evaluate shoulder movement. In addition, markerless human motion capture could also be a potential tool for studying body biomechanics. Recently, it was developed a new computer vision technology for markerless pose estimation, named DeepLabCut (DLC), based on transfer learning with deep neural networks. This study evaluated the validity of DeepLabCut contactless method in measuring active shoulder movements in CSCI patients, while seated in a wheelchair, in a clinical setting. To achieve this, we compared the accuracy of the DeepLabCut contactless method to a customized, wireless wearable IMU-based sensors system. Methods: Eight CSCI patients and 8 healthy controls performed four shoulder movements (forward flexion, abduction, external and internal rotation at 90 of shoulder abduction) with dominant upper limb. Every movement was recorded by IMU system and 2 cameras (GoPro Hero 5) placed frontally and laterally to the subject at the same time. Two IMU sensors of the MbientLab (MetaMotion R), characterized by small dimensions and low cost, were positioned on the arm, just above the elbow, and on the wrist, respectively. For each trial, joints center locations were manually applied to 10 images from each video, and were used as training data for the neural network (ResNet-101), which is in line with recommendations when using DLC. After training, all the videos were analyzed by the DLC and the predicted joints center locations during shoulder movements were extracted. Finally, angle measurements of the extracted coordinates of the joints were calculated with Matlab R2019b. Results: DLC reliably tracked and predicted joint center pixel locations from video recordings. Shoulder ROM measurements of DLC correlated well by comparing with IMU sensors system. Conclusion: DeepLabCut, a new technology for markerless pose estimation, can quantify shoulder ROM measurements in CSCI patients and healthy subjects. From a conventional video recording, DLC allows for objective contactless measurements and this open up possibilities to build tele-rehabilitation.