Photoelectric angle encoders, working as position sensors, have a great influence on the accuracy and stability of telescope control systems (TCS). In order to improve the tracking precision of TCS, a method based on subdivision error compensation for photoelectric angle encoders is proposed. First, a mathematical analysis of six types of subdivision errors (DC error, phase error, amplitude error, harmonic error, noise error, and quantization error) is presented, which is different from the previously used analysis based on the Lissajous figure method. In fact, we believe that a mathematical method is more efficient than the figure method for the expression of subdivision errors. Then, the distribution law and period length of each subdivision error are analyzed. Finally, an error compensation algorithm is presented. In a real TCS, the elevation jittering phenomenon occurs, which indicates that compensating for the amplitude error is necessary. A feed-forward loop is then introduced into the TCS, which is position loop- and velocity loop-closed, leading to a decrease of the tracking error by nearly 54.6%, from 2.31” to 1.05”, with a leading speed of 0.25°/s, and by 40.5%, from 3.01” to 1.79”, with a leading speed of 1°/s. This method can realize real-time compensation and improve the ability of TCS without any change of the hardware. In addition, independently of the environment and the kind of control strategy used, this method can also improve the tracking precision presumably because it compensates the measuring error inside the photoelectric angle encoder.