When the non-standard customized brush roller tool is used for robotic grinding of large-scale components, the clamping and positioning error of the brush roller at the end of the robot is extremely easy to cause misalignment at the brush roller - workpiece contact interface, which will affect the machining accuracy and surface quality. In order to ensure the parallel contact between the brush roller and the workpiece surface during the machining process, a calculation model of the angular misalignment at the brush roller - workpiece contact interface is proposed based on the elastic contact force perception, and then the accurate positioning of the robot end brush roller is realized by a fast compensation method. Firstly, according to the geometric force relationship between the brush roller and the workpiece, as well as the determined brush roller material properties parameters, the estimation model of angular misalignment is established. Secondly, both the axial force and normal torque at the time of initial contact detected by the force-controlled sensor are regarded as the input parameters in the model. Further, the calculated brush roller - workpiece contact offset is used as the geometric error compensation amount, and the brush roller is deflected to achieve error compensation by the robot RAPID program control command. The finite element simulation results are compared with the theoretical calculation values, and the average relative error is 15.1%. The experiment on robotic grinding and brushing of high-speed rail body indicates that the compensated angle can be reduced to 0.024° from an average of 0.179° before compensation, coupled with uniform material removal depth. The proposed method can significantly improve the contour accuracy of large-scale components.