Probabilistic Reasoning with Transformer Networks: An Application to Predicate Judgment

Transformer networks that are trained on participant-generated feature norms have been shown to replicate several behavioral patterns observed in prior research on human semantic cognition. In this paper, I compare the predictions of these networks with participant evaluations of sentences sampled from a vast domain of discourse, with thousands of naturalistic predicates. I find that existing transformer networks make human-like predictions when given atomic sentences. However, they fail to capture participant evaluations of compound sentences with conjunctions, disjunctions, negations, and conditionals. Instead, participant responses are best captured by a Probabilistic Reasoner, which aggregates transformer judgments according to the rules of probability theory. These results suggest that human semantic cognition may also rely on a combination of associative processes and deliberative processes when judging complex statements about the world. In doing so they show how probabilistic cognitive modeling and neural network modeling can be integrated to study naturalistic high-level cognition.