Behavioral Markers of Psychotic Disorders: Using Artificial Intelligence to Detect Nonverbal Expressions in Video

Emotional deficits in psychosis are prevalent and difficult to treat. In particular, much remains unknown about facial expression abnormalities, and a key reason is that expressions are very labor-intensive to code. Artificial intelligence measures of non-verbal expressions (nveAI) can remove this barrier. The current study sought to increase understanding of facial expression abnormalities in psychotic disorders by using nveAI. Changes of facial expressions and head position of participants—39 with schizophrenia/schizoaffective disorder (SZ), 46 with other psychotic disorders (OP), and 108 never psychotic individuals (NP)—were assessed via FaceReader, a commercially available automated facial expression analysis software, using video recorded during a clinical interview. We first examined the behavioral markers of the psychotic disorder groups and tested if they can discriminate between the groups. Next, we evaluated links of behavioral markers with clinical features (symptoms, functioning, and physical performance) controlling for group membership. We found the SZ group was characterized by significantly less variation in neutral expressions, happy expressions, arousal, and head orientation compared to NP. These markers discriminated SZ from NP well (AUC=.79, sensitivity=.79, specificity=.67) but discriminated SZ from OP less well (AUC=.66, sensitivity=.77, specificity=.46). We also found significant correlations between clinical features and all behavioral markers (particularly happy expressions, arousal, and head orientation), except disgust. Taken together, these results suggest that nveAI can provide useful behavioral markers of psychosis, which could improve research on non-verbal expressions in psychosis and, ultimately, enhance treatment.