Challenges and Outlook to Designing Cutting-Edge Mixed Reality Technologies of Human Pose Estimation

Algorithms for real-time tracking of human motion allows projects to be economical and applicable in clinics although less accurate than the gold standard VICON system. Besides, these systems may implement appli-cations more user-friendly and without specific skills needed so clinicians and elderly could use them. The integration of this category of systems with Virtual (VR) or Mixed Reality (MR) will support diagnostic and follow up protocol in telemedicine and teleconsulting, even more for patients with spe-cific diseases that affect movement and joint function which request specific rehabilitation protocol. Student training could benefit from these applications through the interaction with upper and lower limbs and visualization of how diseases affect anatomical movements. The goal is to obtain a comprehen-sive framework for camera-based algorithms of human pose estimation cur-rently used in the medical field, especially in telerehabilitation. The suggest-ed framework comprises 3 layers: the first layer considers pros and cons (limits) of using; the second focuses on possibility in Unity project imple-mentation and the third layer is the integration in VR or MR. This break-through can be considered as a proper resource for the future development of a complete and efficient project in VR or MR choosing the suitable tool to integrate human tracking algorithms in headsets without the use of external webcams or devices.