Hand Gesture Recognition Exploiting Handcrafted Features and LSTM

Hand gesture recognition finds application in several heterogeneous fields, such as Human-Computer Interaction, serious games, sign language interpretation, and more. Modern recognition approaches use Deep Learning methods due to their ability in extracting features without human intervention. The drawback of this approach is the need for huge datasets which, depending on the task, are not always available. In some cases, handcrafted features increase the capability of a model in achieving the proposed task, and usually require fewer data with respect to Deep Learning approaches. In this paper, we propose a method that synergistically makes use of handcrafted features and Deep Learning for performing hand gesture recognition. Concerning the features, they are engineered from hand joints, while for Deep Learning, a simple LSTM together with a multilayer perceptron is used. The tests were performed on the DHG dataset, comparing the proposed method with both state-of-the-art methods that use handcrafted features and methods that use learned features. Our approach overcomes the state-of-the-art handcrafted features methods in both 14 and 28 gestures recognition tests, while we overcome the state-of-the-art learned features methods for the 14 gesture recognition test, proving that it is possible to use a simpler model with well engineered features.