To achieve high accuracy and precision in optical metrology for advanced semiconductors, it is crucial to identify and compensate for errors from optical components and environmental perturbations. In this study, we investigated the sources of the errors in the interferometric ellipsometer developed for next-generation OCD. The objective lens and beam splitters, the critical optical components of the system, are intensively investigated. The system errors induced by temperature fluctuation, wavelength inaccuracy, and defocus were quantitatively examined. We also proposed methods for compensating individual errors and analyzed the effect of the compensation. As a result of error compensation, the accuracy and precision of the system is improved by 6.9 times and 2.3 times, respectively. Although the investigation was conducted based on our interferometric ellipsometry system, the finding is not limited to this system, as these errors are commonly found in most optical metrology systems. The proposed method for error compensation will be essential strategies for various ellipsometry systems suffering from a low level of accuracy and precision.