Research on video action recognition based on Tri-Element Comprehensive Perception Attention

Abstract When pursuing efficient methods for video action recognition, compared to target detection tasks, the recognition of actions and the temporal sequence of information are intricately connected. So, it often requires more complex models. Additionally, the recognition of action typically involves multiple feature dimensions. How to capture the correlations between different feature dimensions effectively is important for improving the accuracy of action recognition. To face these challenges, we propose a lightweight and effective module called the Tri-Element Comprehensive Perception Attention (TCP) module, which contains three parts: Channel Cross Attention, Spatial Fusion Attention, and Motion Perception Attention. These parts respectively learn information in the channel dimension, spatial dimension, and motion feature dimension through videos, and capture the inter-dimensional correlations effectively. Through extensive experiments, we verify the effectiveness of TCP and its sub-modules in terms of video-level accuracy and frame-level accuracy on the UCF101 and HMDB51 datasets. For example, using 3D Resnet as the backbone network, TCP attention achieves an accuracy of 93.13% and 70.46% on UCF101 and HMDB51 datasets, with improvements of 3.53 and 8.06 percentage points compared to the original model. Compared to advanced methods in this field, TCP demonstrates superior performance in action recognition tasks.