Video Based Hand Gesture Detection System Using Machine Learning

Abstract

Machine Learning and convolution networks have achieved great success in image recognition. However, for action recognition in videos, their advantage over traditional methods is not so evident. In this paper, a general and flexible video-level framework for learning action models in videos is presented by considering the Temporal Shift Module (TSM) and Convolutional Neural Networks (CNN) Using Jetson Nano KIt. The main objective is to develop a prototype for implementing a low latency and highly efficient real-time online hand gesture recognition system with low computational cost. The problem is very challenging when it comes to processing speed and efficiency/accuracy. To solve this problem, we are using a multi-modal algorithm (TSM+2D CNN). In this algorithm, TSM combines with 2D Convolutional Neural Networks (CNN) to achieve high efficiency. In the prototype we trained a hand action recognition model capable of detecting around 15 different actions from online videos which can be an efficient and natural human-computer interaction. The design procedure of the trained model has been explained completely. The result obtained in the design and implementation of the system has been quite satisfactory.