EMRNet:Enhanced Micro-Expression Recognition Network with Attention and Distance Correlation

Abstract Micro-expression recognition (MER) is challenging because extracting locally subtle changes in micro-expressions (MEs) is extremely difficult. Several optical flow-based methods have been proposed to recognize MEs because optical flow can effectively suppress facial identity information while characterizing ME movements well. However, these approaches with simple structures do not reveal discriminative features, which leads to inferior performance. This paper proposes an Enhanced Micro-expression Recognition Network with attention and distance correlation (EMRNet) for MER. Concretely, EMRNet involves three phases. First, two identical structural Inception networks with channel-wise region-aware attention are designed to learn parallel global and local expression features based on the same ME’s optical flow input. Second, to empower ME representations, a dilated loss incorporating distance correlation is proposed to amplify the information entropy transmitted by the two branches. Last, emotion categories are predicted via fusing expression-dilated features in the classification branch. Extensive experiments conducted on the composite database published by MEGC 2019 validate the effectiveness of EMRNet under leave-one-subject-out (LOSO) cross-validation and composite database evaluation (CDE) protocol. The results indicate that our proposed approach can generate discriminative features and yield promising performance gains. Moreover, the results also show that EMRNet achieves superior performance and outperforms existing simple single-stream and dual-stream models for MER.