The tilt tri-rotor unmanned aerial vehicle (UAV) has three flight modes: the hover mode, the transition mode, and the fixed-wing mode. Controller design in the hover mode is the premise of realizing stable flight of this kind of UAV. Due to the particular structure with odd rotors and strong nonlinearity, the modeling and control of the tilt tri-rotor UAV remain an active and ongoing research topic. To overcome these problems and achieve stable flight control, this paper proposes a sliding mode-based nonlinear control scheme for the hovering flight of a tilt tri-rotor UAV, consisting of position control, attitude control, and control allocation. First, the mathematical model of the UAV is given by using the Newton–Euler formulation. Second, a cascade flight controller consisting of the position controller and the attitude controller is developed based on sliding mode control (SMC). For the position controller, an auxiliary dynamic system composed of the hyperbolic tangent functions is introduced to the SMC approach for constraining the output magnitude of the thrust and the reference attitudes. Besides, a disturbance observer is applied to the attitude controller to alleviate the chattering and improve robustness. Furthermore, according to the structural characteristics of the tilt tri-rotor UAV, a control allocation algorithm is developed to map the virtual control quantities calculated by the cascade flight controller to the actual actuators. Simulations are conducted to verify the robustness against the external disturbances and parameter variations, and the performance comparisons with two other control schemes are also given. Finally, the experiment is also carried out to validate the performance of the proposed control scheme.