In this paper, a new swarm intelligent-based backstepping control scheme is proposed for quadrotor trajectory tracking and obstacle avoidance. First, the sliding mode extended state observer (SMESO) is used to estimate different disturbances, and the tracking differentiator (TD) is integrated to enhance the performance of backstepping control scheme. Then, the chaotic grey wolf optimization (CGWO) is developed with chaotic initialization and chaotic search to optimize the parameters of attitude and position controllers. Further, the virtual target guidance approach is proposed for quadrotor trajectory tracking and obstacle avoidance. Comparative simulations and Monte Carlo tests are carried out to demonstrate the effectiveness and robustness of the CGWO-based backstepping control scheme and virtual target guidance approach.