Turing Jest: Do Large Language Models have a Sense of Humor?

Humor is an essential aspect of human experience, yet surprisingly little is known about how we recognize and understand humorous utterances. Most theories emphasize the role of incongruity detection and resolution, as well as cognitive capacities like Theory of Mind or pragmatic reasoning. In multiple pre-registered experiments, we ask whether the ability to understand verbal humor can emerge from exposure to purely linguistic input. We find that GPT-3, a large language model (LLM) trained on only language data, exhibits above-chance performance in tasks designed to detect, appreciate, and comprehend jokes. Although GPT-3 falls short of human performance, both humans and LLMs misclassify non-jokes with surprising endings as jokes. Further exploratory analyses reveal a relationship between model size and humor comprehension ability. Results suggest first, that LLMs are surprisingly adept at humor comprehension, and second, that language is not all one needs to “get the joke”.