To handle the disturbances and uncertainties issue in the control of the stabilized platform toolface angle, a composite control approach based on a backstepping sliding mode control (BSMC) method with a nonlinear disturbance observer (NDO) is proposed. First, a nonlinear mathematical model of the stabilized platform is established, and the NDO is employed to observe and compensate for the external disturbance. Then, the BSMC control approach based on the nonlinear mathematical model is designed to improve the robustness of the stabilized platform control system. The asymptotic stability of the designed controller and NDO convergence are mathematically verified using the Lyapunov theory. Furthermore, to address the dead zone torque caused by mechanical friction on the stabilized platform, which could lead to instability in the toolface angle, a novel dead zone torque estimation algorithm is presented. Finally, in comparison with other advanced control methods, simulation results show the superiority and robustness of the proposed scheme under complex disturbances from the downhole environment. And drilling simulation experiment results are provided to validate the robustness and adaptivity of the proposed method.