Relating visual and pictorial space: Binocular disparity for distance, motion parallax for direction

Interacting with people and three-dimensional objects depicted on a screen is perceptually different from interacting with them in real life. This difference resides in their corresponding perceptual spaces: The former involves pictorial space, and the latter, visual space. Studies have examined the perceptual geometry of pictorial or visual space, but rarely their connection. In the current study, we connected visual and pictorial space using an exocentric pointing task and investigated how binocular disparity and motion parallax affect this connection. In a virtual environment, we displayed a pointing virtual character within a frame and asked participants to rotate him to point at targets located in visual space. We independently manipulated what binocular disparity and motion parallax specified inside the frame, either the two-dimensional surface or its depicted three-dimensional content. In Experiment 1, we manipulated the virtual character’s distance to the screen and found that binocular disparity determines the distance relationship between visual and pictorial space, but it also introduces a relief depth expansion of the perceived virtual character. In Experiment 2, we changed the participants’ viewing angle relative to the screen and found that motion parallax determines the directional relationship between visual and pictorial space. We discuss the theoretical and practical implications of our results in the context of video-mediated telecommunication.