The optical system of the laser tracker utilizes plane mirrors to construct a reflective path, reducing its size and weight. However, maintaining the alignment of the laser with the ideal optical axis during its propagation in the optical system poses significant challenges in the design, fabrication, and assembly of the optical system. This paper explores the principle of error propagation during the assembly process of the optical system and improves the accuracy of the output laser through a numerical simulation and optimization methods. A general error model for the optical system is established to understand the principle of error propagation. A Monte Carlo numerical simulation and sensitivity analysis are used to study the influence of various errors on the accuracy of the output laser. The machining errors are optimized using a simulated annealing method to balance the manufacturing difficulty and system accuracy. The assembly process is also optimized to reduce the degrees of freedom and the number of optical parts required, and verified by experiments. The experimental results indicate that the average position error of the output laser is 15.743 µm, and the average angle error is 1.427′′. This study provides what we believe is a novel approach and methodology for the design and alignment of optical systems.