Reliable and Valid Rating Data in Less Time with the Fast Image Rating Experiment

Researchers often assess processes underlying human perception by measuring participants’ judgements of image stimuli. However, traditional methods for quantifying subjective judgements, such as Likert scales, sliding scales, and pairwise comparisons, are vulnerable to biases or demand extensive time and resources from researchers and participants. The present study compared the efficiency, reliability, and validity of these established methods against the Fast Image Rating Experiment (FIRE), our force-choice-based paradigm for assessing perceptions of visual stimuli. When used to rate image preference and naturalness, the FIRE was five times faster than established methods, highly reliable, and valid. FIRE achieved high reliability in less than half the time required to reach equivalent reliability with the Likert or sliding scale, which could save researchers thousands of dollars. The scalability and cost-effectiveness of the FIRE make it a valuable resource for supporting large-scale behavioral science.