In the process of large aspheric optical surfaces fabrication, the distortion of the removal function is a big problem that affects the producing efficiency and accuracy, due to the misfit between the tool and the aspheric surface in the contact region. Consequently, this paper aims to find out the influence factors and the distortion rule of aspheric removal function in the computer-controlled optical surfacing. Firstly, based on the analysis of the sub-aperture polishing technology for the large aspheric optical surfaces, the local asphericity of aspheric surface and the viscoelasticity of polishing tool are supposed to be the main sources. After that, a method to calculate the local asphericity considering the misfit between the tool and the aspheric surface is proposed based on the least square method, and the viscoelasticity of the polishing tool is obtained through viscoelastic experiment. Subsequently, combining the results of the local asphericity of aspheric surface and the viscoelasticity of polishing tool, the prediction of the distortion rule of aspheric removal function is presented. Finally, the comparative experiment is carried out, and the removal function on different regions of the aspheric surface is obtained. The experimental result indicated that the distortion of the removal function is consistent with the theoretical result. Through this study, the distortion rule of aspheric removal function in the computer-controlled optical surfacing with pitch tool is finally mastered, which provides a theoretical guidance for the computer-controlled optical surfacing process optimization.