This paper devotes to addressing the distributed containment formation control problem for multi-UAVs with collision avoidance and external disturbances. The proposed communication structure design algorithm enables the followers to form the pre-defined formation based on the containment control. Then, based on the information of the desired position for the followers, a novel Lyapunov function is designed to achieve global collision avoidance, and an adaptive backstepping containment control law is proposed. Moreover, by taking the advantage of deep reinforcement learning, a parameter optimization method is presented to balance the value of input signals and the performance of the controller. Finally, the simulation results demonstrate the superiority and effectiveness of the proposed algorithms.