Single-stage prediction models do not explain the magnitude of syntactic disambiguation difficulty

The disambiguation of a syntactically ambiguous sentence in favor of dispreferred parse can lead to slower reading at the disambiguation point. This phenomenon, referred to as a garden path effect, has motivated models in which readers only maintain a subset of the possible parses of the sentence; reverting to a discarded parse requires costly reanalysis. More recently, it has been proposed that the garden path effect can be reduced to surprisal arising in a fully parallel parser: words consistent with the initially dispreferred but ultimately correct parse are simply less predictable than those consistent with the incorrect parse. The surprisal account is more parsimonious since predictability has pervasive effects in reading far beyond garden path sentences. Crucially, this account predicts a linear effect of surprisal: the difficulty experienced by readers should be proportional to the difference in word surprisal between the ultimately correct and ultimately incorrect interpretations. To test this prediction, we estimated word-by-word surprisal using recurrent neural network language models, comparing those estimates to self-paced reading times for three garden path constructions. While surprisal successfully predicted the existence of garden path responses, it severely underpredicted the magnitude of all of the garden path effects. Further, the relative size of the predicted effects was inconsistent with the relative size of the responses in humans, indicating that a differently scaled linking function would not be able to predict the response magnitudes either. These results support two-stage processing models in which recovery mechanisms beyond predictability are involved in processing garden path sentences.