Expectation management in humans and LLMs

Abstract Mirative markers, such as “surprisingly”, explicitly encode a violation of expectations. Such markers are used for expectation management during communication. Sensitivity to mirative markers relies on two abilities: i) updating expectations upon recognizing a mirative marker, and ii) identifying expectation violations warranting the use of a mirative marker. In this study, we compared sensitivity to mirative markers in humans and large language models (LLMs). In part 1, we used a sentence-completion task, where humans and LLMs were presented with sentence fragments and asked to continue them. Results show that for both humans and LLMs, the presence of a mirative marker significantly increased response entropy and decreased top-response probability, in line with theoretical accounts of mirative processing. In part 2, we created a novel task, mirative polarity selection, where humans and LLMs are presented with a sentence pair and asked to select whether it was connected by a mirative marker (“surprisingly”) or an anti-mirative marker (“unsurprisingly”). Results show that LLMs perform at an impressive human-equivalent level. We conclude that both humans and LLMs use mirative markers as cues for calibrating their subsequent expectations during sentence processing. Competing Interest Statement The authors have declared no competing interest. Footnotes fixed minor typos; few sentences edited for clarity.