Classic control algorithm can improve dynamic performance of opto-electronic tracking system, but the tracking system often has little robust performance. From the standpoint of inproving robust performance, modern robust control technology was adopted to design the controller of opto-electronic tracking system in this paper. On the basis of considering the system tracking precision requirement and main disturbance, the mathematical model of the system augmented objict was established. By repeatly testing, the suitable weighted functions were selected. Finally, the robust controller of opto-electronic tracking system was designed by synthesis method of modern robust control theory. The simulation result shows that the adjusting time of the system is 0.3S, and it can satisfy the requirements of the tracking speed. Robust controller has improved the system dynamic response characteristics and makes the system fast and no overshoot, and the opto-electronic tracking system has stronger robust stability and robust performance.