Illumination estimation is crucial for surface detection method based on visual image reconstruction. The existing illumination models which depend on the macro-surface geometric features and reflection characteristics are difficult to accurately describe the luminance of machined surface topography with desertification fractal structure. Therefore, machined surface topography illumination model under coaxial light microscopic vision is proposed in this paper for the needs of on-line detection of complex surface and special functional surface topography in manufacturing field. Based on the light scattering theory, the luminance of surface topography is studied under the coaxial light microscopic vision, the influencing factors of luminance of surface topography are revealed, and the illumination model of machined surface topography is established. The experimental results show that the proposed illumination model has less calculation error and can more accurately describe the luminance of machined surface topography. The research results will lay a foundation for on-line detection and monitoring of machined surface.