Vertical takeoff and landing (VTOL) is an essential feature of unmanned aerial vehicles (UAVs). On the one hand, VTOL can expand and enhance the applications of UAVs; yet, on the other hand, it makes the design of control systems for UAVs more complicated. The most challenging demand in designing the control system is to achieve satisfactory response sharpness of fixed-wing UAVs to control commands and ensure that the aircraft mode channels are effectively decoupled. In this work, a six-degree-of-freedom (6-DoF) model with forces and moments is established based on the aerodynamic analysis, which is carried out through computational fluid dynamics (CFD) numerical simulation. The improved proportional derivative (PD) controller based on the extended state observer (ESO) is proposed to design the inner-loop attitude control, which increases the anti-interference ability for internal and external uncertainty of the UAV system. The motion equations of the UAV are established and divided into independent components of longitudinal and lateral motion to design the outer loop control law under minor disturbance conditions. A total energy control system (TECS) for the longitudinal height channel is proposed, which separates speed control and track control. L1 nonlinear path tracking guidance algorithm is used for lateral trajectory tracking so as to improve curve tracking ability and wind resistance. Effectiveness of this approach is proved by actual flight experiment data. Finally, a controller based on angular velocity control is designed to prevent the attitude and head reference system (AHRS) from malfunctioning. Its effectiveness is verified by the response test of the control system.