The spatial resolution of the human visual field decreases considerably from the center to the periphery. However, several studies have highlighted the importance of peripheral vision for scene categorization. In Experiment 1, we investigated if peripheral vision could influence the scene categorization in central vision. We used photographs of indoor and outdoor scenes from which we extracted a central disk and a peripheral ring. Stimuli were composed of a central disk and a peripheral ring that could be either semantically congruent or incongruent. Participants had to categorize the central disk while ignoring the peripheral ring or the peripheral ring while ignoring the central disk. Results revealed a congruence effect of peripheral vision on central vision, as strong as the reverse. In Experiment 2, we investigated the nature of the physical signal in peripheral vision that influences the categorization in central vision. We used either intact, phase-preserved, or amplitude-preserved peripheral rings. Participants had to categorize the central disk while ignoring the peripheral ring. Results showed that only phase-preserved peripheral rings elicited a congruence effect as strong as the one observed with intact peripheral rings. Information contained in the phase spectrum (spatial configuration of the scene) may be critical in peripheral vision.