Using natural language processing to assess the impact of cognitive reappraisal

Cognitive reappraisal is one of the most widely studied emotion regulation strategies. Recent research suggests that reappraisal involves shifting appraisals of situations along multiple dimensions. However, assessing these shifts is challenging as individuals can find it difficult or cumbersome to report upon them directly. We sought to determine whether natural language processing (NLP) methods, including large language models (LLMs), supplement self-reports of appraisals. We conducted an experiment with young adults (Study 1: n = 1943) that we replicated in a pre-registered experiment with adults (Study 2: n = 2449). Participants were randomized to regulate a recent upsetting situation via (1) cognitive reappraisal or (2) an active control (expressive writing). Self-reports of distress and appraisals were captured pre- and post-regulation. Text descriptions of the situation were captured pre- and post-regulation and analyzed with the Sentiment Analysis and Cognition Engine (SEANCE) as well as the LLM LLAMA 3. Across both samples, cognitive reappraisal (versus expressive writing) led to greater improvements in self-reported distress and changes in most appraisals. Cognitive reappraisal led to changes in the SEANCE metrics, but most of the effects were small and many were inconsistent across the samples. By contrast, cognitive reappraisal led to greater LLM-rated cognitive changes and overall changes in textual similarity and these were consistent across both samples. Self-report and NLP indices capture the process of cognitive reappraisal. In particular, the LLM reliably detected overall text changes as well as changes in cognition.