For compensating backlash phenomenon in servo systems, the authors propose an observer method in this paper to estimate both system states and vibration torque before controller design. First, a systematic scheme is given to obtain plant parameters, which is very important in observing system states. This is a parameter estimation principle that gives a crude estimation and computes the differences between the crude and true values. As a result, the precise value of the parameters is obtained by adding together the crude value and the difference. Then, based on the precise estimated parameters, an extended state observer (ESO) is designed to obtain feedback and feedforward signals. Consequently, robust compensation control is achieved by designing an output feedback controller, consisting of a feedback term and a feedforward term. Finally, in order to validate the proposed approach, extensive experiments are performed on a practical servo system with backlash nonlinearity.