Using artificial neural networks to relate external sensory features to internal decisional evidence

All theories of perceptual decision-making postulate that external sensory information is transformed into the internal evidence that is used to guide behavior. However, the nature of this external-to-internal transformation is generally unknown. In two experiments, we examined how a particular stimulus feature–orientation–is transformed into internal evidence. Subjects judged whether Gabors were tilted clockwise or counterclockwise. The results of Experiment 1 demonstrated that increasing the stimulus tilt in fine-scale increments resulted in a linear increase in sensitivity (d’), suggesting a linear external-to-internal transformation. However, the results of Experiment 2 demonstrated that increasing the stimulus tilt in coarse-scale increments had little effect on sensitivity, suggesting a highly non-linear transformation. These results suggest that external sensory information is transformed into internal decisional evidence in complex ways. Critically, artificial neural networks (ANNs) trained on the orientation task and validated against the empirical results provided a framework for examining how sensory stimuli map onto internal evidence. The hidden-layer activations of the ANNs revealed that fine-scale increments in tilt magnitude results in increasingly greater discriminability between the stimulus categories, but the degree of discriminability does not increase further after the magnitude of stimulus tilt becomes sufficiently large. Taken together, these results begin to reveal how external sensory information is transformed into the internal evidence that is used to make decisions and suggest that ANNs could serve as a platform for understanding the mechanism underlying this critical transformation.