What’s wrong with the reference group effect in cross-cultural research?

Cross-cultural researchers often compare mean scores across cultures using subjective Likert scales, but this approach faces several methodological challenges. One alleged challenge is the reference group effect (RGE), which posits that differing normative standards distort cultural comparisons, potentially suppressing or reversing true differences. In this study, we revisit this widely discussed yet under-examined issue and test its premise. Leveraging data from the International Social Survey Programme and the World Values Survey, we examined the relationship between subjective socio-economic status (SES) and its external criterion, national wealth (GDP). According to the RGE, respondents compare themselves to their peers within their cultural norms. This would result in individuals from wealthy countries rating themselves as less affluent and those from developing countries rating themselves as relatively more affluent. Such biased comparisons would suppress true differences, leading to a null correlation between SES and GDP across countries. However, our findings consistently revealed a strong positive correlation: respondents from wealthier countries rated themselves higher, while those from developing countries rated themselves lower. Contrary to the RGE, respondents appear to interpret numerical anchors in a single, universal, and absolute manner—“Upper class” carries the same meaning regardless of varying economic references. This consistent interpretation enables valid cross-cultural differences to emerge. Our findings challenge the validity of the RGE and encourage cross-cultural researchers to interpret observed cultural differences with greater confidence at face value when using Likert scales.