Intertemporal Differences Among MTurk Workers

The online labor market Amazon Mechanical Turk (MTurk) is an increasinglypopular platform for generating samples of respondents for social scienceresearch. A growing body of research has examined the demographiccomposition of MTurk workers, typically comparing samples of MTurk workersto samples of respondents drawn from other populations. While thesecomparisons have revealed important information about the ways in whichMTurk workers are and are not representative of the general population,variations among samples drawn from MTurk have received less attention.This paper focuses on whether MTurk sample composition varies as a functionof time. Using an original dataset of nearly 10,000 MTurk workers, weexamine whether demographic characteristics vary by (1) time of day, (2)day of week, and (3) serial position (i.e., earlier or later in datacollection). We find that day of week differences are minimal, but thattime of day and serial position are associated with small but importantvariations in demographic composition, including characteristics known toimpact political attitudes and psychological processes. This demonstratesthat MTurk samples cannot be presumed identical across different studies,and we suggest several forms of variation to which researchers using MTurkought to be attentive.