A Lightweight Model for Accurate Multi-View Hand Pose Recognition

This paper introduces a lightweight architecture for multi-view hand pose recognition on multimodal fusion of images and landmarks. The proposed model employs a compact Convolutional Neural Network (CNN) to extract visual features from dual-view grayscale images, while a Multi-Layer Perceptron (MLP) processes the corresponding Leap Motion Controller 2 hand landmarks. The two modalities are fused to create an efficient yet discriminative representation. Compared to the Vision Transformer (ViT)+MLP baseline, which achieves an F1 score of 79.33 ± 0.09 % with 8.95 × 107 parameters, our CNN+MLP model reaches a higher recognition accuracy of 85.36 ± 0.08 % while requiring only 2.13 × 105 parameters, which corresponds to an important reduction of the model size. Moreover, a landmarks-only variant using the MLP achieves 85.22 ± 0.08 % accuracy with just 6.46 × 104 parameters. These results, obtained on the Multi-view Leap2 Hand Pose Dataset under a Leave-One-Subject-Out Cross-Validation protocol, demonstrate that accurate multi-view hand pose recognition can be achieved with dramatically fewer parameters, enabling efficient deployment in resource-constrained environments.