This article investigates the consensus tracking control problem of the leader-follower spacecraft formation system in the presence of model uncertainties, external disturbances, and actuator saturation, where the relative motion of the leader and the follower need to track a desired time-varying trajectory given in advance. First, the six-degree-of-freedom relative-coupled translational and rotational dynamics models are derived using the exponential coordinates on the Lie group SE(3). Then, a fast terminal sliding mode control law is proposed to guarantee the tracking control objective come true in finite time robust against all the aforementioned drawbacks. As a stepping stone, an extended state observer is designed to estimate and compensate the total composed disturbances of the system, and it is proved that the estimate errors can converge to a really small neighborhood of the origin in finite time. Based on the observer information, a less-conservative modified controller is furthermore developed to eliminate the chattering caused by the signum function. The stability of the closed-loop system is shown by theoretical analysis. Finally, the validity of the proposed schemes is illustrated through numerical simulations.