Biomedical Text Readability and Cognitive Burden after Hypernym Substitution with Fine-Tuned Large Language Models

Abstract We aimed to study the simplification of biomedical text via large language models (LLMs). Specifically, we finetuned three language models to perform substitutions of complex words and word phrases for their respective hypernym in biomedical definitions. This process was then evaluated by readability metrics, and two measures of sentence complexity: the measure of lexical diversity (MLTD), and mean dependency distance (MDD) scoring. A sample of 1,000 biomedical definitions in the National Library of Medicine’s Unified Medical Language System (UMLS) was processed with three approaches, each with a different language models and analysis revealed an increase in FK score and a reduction in reading grade level across all metrics. Reading scores improved from a pre-processed collegiate reading level to a post-processed US high-school level. An inter-approach comparison showed that our GPT-J-6b approach had the best improvement in MLTD and MDD. This study demonstrates the merit of hypernym substitution in improving the readability and improving measures cognitive burden of biomedical content for the general public.