The hole parallelism error (HPE) is one of the key quality parameters for mechanical parts. However, HPE can usually be out-of-tolerance due to the coupling effect of residual stress, cutting stress, and geometric structure during its machining process. Thus, on-machine measurement and timely adjustment are important guarantees for HPE. This paper proposes an on-machine measurement method of HPE based on multi-sensor fusion that is low-cost, efficient, and suitable for mass production of parts. First, the axis of the cylindrical surface is directly extracted using five points according to the spatial line position relation. Then, the mathematical model of the HPE of two holes’ axes was established through the theory of spatial analytic geometry. The HPE is characterized using two parameters: parallelism magnitude and parallelism direction. Finally, the feasibility of the measurement method in principle and practical application is verified by simulation and experiment. The simulation tests show that the maximum value of the relative error of parallelism magnitude and parallelism direction is −1.38‱. The actual experiments show that the relative error of the measurement results compared with the coordinate measuring machine is 6.23%. None of the standard deviations exceeds the order of magnitude of 0.01 mm. The difference in measurement uncertainty is 0.0003 mm.