Don’t Keep It Too Simple: Simplified Items Do Not Improve Measurement Quality

When formulating questionnaire items, generally accepted rules include: Keep the wording as simple as possible and avoid double-barreled items. However, the empirical basis for these principles is sparse. The present study aimed to systematically investigate in an experimental design whether simplifying items of a personality scale and avoiding double-barreled items markedly increases psychometric quality. Specifically, we compared the original items of the Big Five Inventory-2—most of which are either double-barreled or can be regarded as complexly formulated—with simplified versions of the items. We tested the two versions based on a large, heterogeneous sample (N = 2,234). The simplified versions did not possess better psychometric quality than their original counterparts; rather, they showed weaker factorial validity. Regarding item characteristics, reliability, and criterion validity, no substantial differences were identified between the original and simplified versions. These findings were also replicated for the subsample of lower-educated respondents, who are considered more sensitive to complex item formulations. Our study thus indicates that simplifying item wording and avoiding double-barreled items does not improve the quality of a questionnaire; rather, using simpler but less specific item formulations decreases factorial validity.