Emotion recognition from gait using multi-scale directed adaptive spatio-temporal graph convolutional network

Graph convolutional network have been widely used for walking emotion recognition. However, the dependencies in the multi-scale,long-distance data and neighbor nodes can affect the efficiency of emotion feature extraction, and further impact the emotion identification accuracy. To this end, we propose i) a multi-scale spatio-temporal information directed aggregation method based on directed graphs, which considers not only the correlation between spatial and temporal features at the same time, but also the directionality of the temporal dimension; ii)a multi-scale directed adaptive spatio-temporal graph convolutional network (MSDAST-GCN), where the features of each spatio-temporal graph node are extracted according to the time span, aggregation scale and size. The MSDAST-GCN extracts motion data from a video, reasons about human emotions and classifies the emotions accurately into four categories: happy, sad, angry and normal. The MSDAST-GCN outperforms previous state-of-the-art methods on the latest dataset E-Gait, shows a 3.5% accuracy improvement. Specifically, the MSDAST-GCN achieves 1.4%, 0.4%, 2.9% and 7.5% accuracy enhancements on the happy, sad, angry and normal categories, respectively.