As an important component of the gravitational wave detection interferometric measurement system, the telescope’s main function is to receive and emit laser signals. Under the influence of in-orbit environmental disturbances, the telescope will experience structural deformation. This can result in changes in the propagation path of the laser within the telescope, which in turn affects the ranging accuracy. Therefore, it is necessary to suppress the optical path variation caused by the deformation of the telescope during the design phase of the telescope structure. In this paper, a calculation and analysis model of the non-geometric optical path length variation within the telescope was established using the first 36 orders of the fringe Zernike polynomials. The derivation of the geometric optical path variation within the telescope was completed, and the optical system error analysis was performed based on the internal optical path variation of the telescope as an evaluation index. The primary mirror components that meet the detection requirements were designed. The results showed that for the off-axis four-mirror optical system, under the same disturbance, the geometric optical path variation caused by the rigid displacement of the primary mirror dominates. When the environmental temperature stability is 2.8×10−6K⋅Hz−1/2 @0.1 Hz, the system’s optical path stability is reduced from the original 8.5pm⋅Hz−1/2 @0.1 Hz to 0.45pm⋅Hz−1/2 @0.1 Hz.