Human Action Detection and Recognition: A Pragmatic Approach using Multiple Feature Extraction Techniques and Convolutional Neural Networks

Abstract Action recognition is described as the capability of determining the action that a human exhibit in the video. Latest innovations in either deep-learning or hand-crafted methods substantially increased the accuracy of action recognition. However, there are many issues, which keep action recognition task far from being solved. The task of human action recognition persists to be complicated and challenging due to the high complexity associated with human actions such as motion pattern variation, appearance variation, viewpoint variation, occlusions, background variation and camera motion. This paper presents a computational approach for human action recognition using video datasets through different stages: Detection, tracking of human and recognition of actions. Human detection and tracking are carried out using Gaussian Mixture Model (GMM) and Kalman filtering respectively. Different feature extraction techniques such as Scale Invariant Feature Transform (SIFT), Optical Flow Estimation, Bi-dimensional Empirical Mode Decomposition (BEMD), Discrete Wavelet Transform (DWT) are used to extract optimal features from the video frames. The features are fed to the Convolutional Neural Network classifier to recognize and classify the actions. Three datasets viz. KTH, Weizmann and Own created datasets are used to evaluate the performance of the developed method. Using SIFT, BEMD and DWT multiple feature extraction technique, the proposed method is called Hybrid Feature Extraction – Convolutional Neural Network based Video Action Recognition (HFE-CNN-VAR) method. The results of the work demonstrated that the HFE-CNN-VAR method enhanced the accuracy of action classification. The accuracy of classification is 99.33% for Weizmann dataset, 99.01% for KTH dataset and 90% for own dataset. Results of the experiment and comparative analysis shows that proposed approach surpasses when compared with other contemporary techniques.