Analysing the Direction of Emotional Influence in Nonverbal Dyadic Communication: A Facial-Expression Study

Identifying the direction of emotional influence in a dyadic dialogue is of increasing interest in the psychological sciences with applicationsin psychotherapy, analysis of political interactions or interpersonal conflict behavior. Facial expressions are widely described as being automatic andthus hard to overtly influence. As such, they are a perfect measure for a better understanding of unintentional behavior cues about social-emotionalcognitive processes. With this view, this study is concerned with the analysis of the direction of emotional influence in dyadic dialogue based on facialexpressions only. We exploit computer vision capabilities along with causal inference theory for quantitative verification of hypotheses on the directionof emotional influence, i.e., causal effect relationships, in dyadic dialogues. We address two main issues. First, in a dyadic dialogue, emotionalinfluence occurs over transient time intervals and with intensity and direction that are variant over time. To this end, we propose a relevant intervalselection approach that we use prior to causal inference to identify those transient intervals where causal inference should be applied. Second, wepropose to use fine-grained facial expressions that are present when strong distinct facial emotions are not visible. To specify the direction ofinfluence, we apply the concept of Granger causality to the time series of facial expressions over selected relevant intervals. We tested our approachon newly, experimentally obtained data. Based on quantitative verification of hypotheses on the direction of emotional influence, we were able to showthat the proposed approach is most promising to reveal the causal effect pattern in various instructed interaction conditions.