Overview of the Advancements in Automatic Emotion Recognition: Comparative Performance of Commercial Algorithms

In the recent years facial emotion recognition algorithms have evolved and in some cases top commercial algorithms detect emotions like happiness better than humans do. To evaluate the performance of these algorithms, the common practice is to compare them with human-labeled ground truth. This article covers monitoring of the advancements in automatic emotion recognition solutions, and here we suggest an additional criteria for their evaluation, that is the agreement between algorithms’ predictions. In this work, we compare the performance of four commercial algorithms: Affectiva Affdex, Microsoft Cognitive Services Face module Emotion Recognition, Amazon Rekognition Face Analysis, and Neurodata Lab Emotion Recognition on three datasets AFEW, RAVDESS, and SAVEE, that differ in terms of control over conditions of data acquisition. We assume that the consistency among algorithms’ predictions indicates the reliability of the predicted emotion. Overall results show that the algorithms with higher accuracy and f1-scores that were obtained for human-labeled ground truth (Microsoft’s, Neurodata Lab’s, and Amazon’s), showed higher agreement between their predictions. Agreement among algorithms’ predictions is a promising criteria in terms of further exploring the option to replace human data labeling with automatic annotation.