In this paper, a robust adaptive sliding mode control scheme is developed for attitude and altitude tracking of a quadrotor unmanned aerial vehicle (UAV) system under the simultaneous effect of parametric uncertainties and consistent external disturbance. The underactuated dynamic model of the quadrotor UAV is first built via the Newton–Euler formalism. Considering the nonlinear and strongly coupled characteristics of the quadrotor, the controller is then designed using a sliding mode approach. Meanwhile, additional adaptive laws are proposed to further improve the robustness of the proposed control scheme against the parametric uncertainties of the system. It is proven that the control laws can eliminate the altitude and attitude tracking errors, which are guaranteed to converge to zero asymptotically, even under a strong external disturbance. Finally, numerical simulation and experimental tests are performed, respectively, to verify the effectiveness and robustness of the proposed controller, where its superiority to linear quadratic control and active disturbance rejection control has been demonstrated clearly.