In classical Fourier optics, an optical imaging system is regarded as a linear space-invariant system, which is only an approximation. Especially in digital holography, the space-variance effect has a great impact on the image quality and cannot be ignored. Therefore, it is comprehensively investigated in this article. Theoretical analyses indicate that the space-variance effect is caused by linear frequency modulation and ideal low-pass filtering, and it can be divided into three states: the approximate space-invariance state, the high-frequency distortion state, and the boundary-diffraction state. Classical Fourier optics analysis of optical imaging systems only considers the first. Regarding the high-frequency distortion state, the closer the image field is to the edge, the more severe the distortion of high-frequency information is. As for the boundary-diffraction state, in addition to the distortion of high-frequency information in the margin, a prominent boundary-diffraction phenomenon is observed. If the space-variance effect of the imaging lens is ignored, we predict that no space-variance effect in image holography will occur when the hologram is recorded at the back focal plane of the imaging lens. Simulation and experimental results are presented to validate our theoretical prediction.