We developed a gaze contingent display in which natural images are presented to the observer with dioptric blur and stereoscopic disparity that are dependent on the 3 dimensional structure of natural scenes. Our system simulates a distribution of retinal blur and depth similar to that experienced in real world viewing conditions by emmetropic observers. We implemented the system using light field photographs taken with a plenoptic camera which supports digital refocusing anywhere in the images. We coupled this capability with an eye-tracking system and stereoscopic rendering. With this display, we examine how the time course of binocular fusion depends on depth cues from blur and stereoscopic disparity in natural images. Our results show that disparity and peripheral blur interact to modify eye movement behavior and facilitate binocular fusion and the greatest benefit was gained by observers who struggled most to achieve fusion. Even though plenoptic images do not replicate an individual’s aberrations, the results demonstrate that a naturalistic distribution of depth-dependent blur may improve 3D virtual reality and interruptions of this distribution, for example with multi-focal lenses, which flatten the distribution of retinal blur may affect binocular fusion.