Human Activity Recognition with Event-Based Dynamic Vision Sensor using Deep Recurrent Neural Network

Abstract

Human activity recognition based on traditional cameras capture video at a fixed frame rate, which is still lacking in speed and starving for high power for a real-time response since a massive number of operations is required to be performed for inference learning using deep learning. The biologically roused event-based Dynamic Vision Sensor (DVS) captures only change in pixel intensity values. This sparse and rapid representation makes event-based cameras operate with low power consumption, low latency, excellent temporal resolution, high dynamic range, and a reduced amount of storage. Moreover, in DVS based cameras only the change in pixel values stored, the privacy of the person can also be preserved. These all attractions move towards the use of dynamic vision sensor cameras for prominent applications in wearable platforms where power hunger is a crucial issue as well as in the applications where the privacy of the person to be preserved. In this paper, we introduce a model for Human Activity Recognition with event-based dynamic vision sensors. Each input frame is pre-processed using image processing methods. The deep convolutional neural network (CNN) extracts the feature of each frame for a given sequence of video input. This feature of each frame is fed to the deep recurrent neural network (RNN) for sequence learning, which is used to recognize human activity. The experimental results on the DVS benchmark dataset display the effectiveness of the proposed model.