Moving beyond word frequency based on tally counting: AI-generated familiarity estimates of words and phrases are a better index of language knowledge

This study investigates the potential of large language models (LLMs) to estimate familiarity of words and multi-word expressions (MWEs). We validated LLM estimates for isolated words using existing human familiarity ratings and found strong correlations. LLM familiarity estimates were found to perform even better in predicting lexical decisions and naming performance in megastudies than the best available word frequency measures. We then applied LLM estimates to MWEs, also finding their effectiveness in measuring familiarity for these expressions.We created a list of over 400,000 English words and MWEs with LLM-generated familiarity estimates, a valuable resource for researchers. There is also a cleaned-up list of nearly 150,000 words, excluding lesser-known stimuli, to streamline stimulus selection.Our findings highlight the advantages of LLM-based familiarity estimates, including their better performance than traditional word frequency measures (particularly for predicting word recognition accuracy), their ability to generalize to MWEs, availability for large lists of words, and ease of obtaining new estimates for all types of stimuli.This work contributes to a better understanding of word recognition and its implications for language processing and reading comprehension.