Detection, Inspection, Return: A functional classification of fixations in complex scenes

Salience models for complex scenes typically aim to explain heatmaps or scan-paths. While heatmaps potentially conflate fixations serving different functions, scan-paths aim for a level of detail which is hard to attain. We introduce a novel approach, based on the premise that most fixations target objects and belong to three functionally meaningful classes: Detection fixations (D), foveating an object for the first time; Inspection fixations (I), successively targeting object details; and Return fixations (R), revisiting an object, e.g., to refresh its representation in memory. To test the hypothesis that these classes capture distinct fixation profiles, we reanalysed a large dataset of scene fixations. We computed separate heatmaps for D, I, and R and found significantly higher inter-observer consistency within than between classes. Across fixations landing on different semantic features, the proportion of D, I, and R fixations varied consistently, and a semantic salience model trained to predict each class independently learned diverging distributions of feature weights. Further, we found a shift from D to I and R across viewing time, in line with their functional interpretation and previous findings on ambient and focal viewing modes. We tested and confirmed that the dynamics of this shift varied as a function of trial duration. Finally, we found that the D, I, R classification could be approximated with a simple spatial heuristic, extending its applicability to images without object metadata. We propose the D, I, and R classification as a simple but powerful tool to functionally classify scene fixations and provide well documented code to implement it.