The Online Volunteer Subject: Who Does and Does Not Participate In Our Studies

Research using online research participants is here to stay. Understanding who these subjects are, and specifically what kind of biases and selection mechanisms they bring to our research studies will be important for the future of human-subjects research as our studies rely more and more on the Online Volunteer Subject. Here we take a method validated to understand biases in in-person research subjects and extend it to the online laboratory. Using this method, we attempt to replicate and extend this work as a tool for other researchers to use in their own online studies to quantify potential biases and space for treatment heterogeneity. Here we show Online participants are different from the rest of the population. The Online Participant is older, more likely to be Jewish, more well educated, more intelligent, less sociable, less arousal-seeking, more conventional, less interested in religion, and less willing to self-disclose than the person who does not take online studies. This systematic bias by focusing on the types of people who take online studies create obvious problems to generalizability. Furthermore, it creates biases in estimation for all types of studies through range restriction on numerous measured (and potentially many more unmeasured) variables. In short, the type of person who participates in our online studies is not just a regular person, they are systematically different. We provide a method to measure this bias in one’s own study.