A one degree of freedom flight motion table is used to simulate the rotational movement of flying objects in hardware‐in‐the‐loop laboratories. This table uses a dual motor driving servo system. Owing to the existence of a gear transmission system, the backlash nonlinear function on modeling is inevitable, system equations are non‐affine and back‐stepping theory cannot handle this kind of system directly. Through new modeling of the backlash function as the summation of certain and uncertain parts, a proper structure for the controller algorithm is created. The controller is constructed through a back‐stepping sliding mode control algorithm. The back‐stepping concept allows the control algorithm to be extended to a high‐order dynamical system. Meanwhile, the SMC could eliminate these system uncertainties as it has a match perturbation with a virtual or real input signal of the subsystem. For non‐measurable variables, a sliding mode observer is also designed. The simulation results clearly demonstrate the superiority of the proposed method in comparison with conventional PID/FO‐PID control methods in angular tracking and speed synchronization.