The problem of collision avoidance of an unmanned aerial vehicle (UAV) group is studied in this paper. A collision avoidance method of UAV group formation based on second-order consensus algorithm and improved artificial potential field is proposed. Based on the method, the UAV group can form a predetermined formation from any initial state and fly to the target position in normal flight, and can avoid collision according to the improved smooth artificial potential field method when encountering an obstacle. The UAV group adopts the “leader–follower” strategy, that is, the leader UAV is the controller and flies independently according to the mission requirements, while the follower UAV follows the leader UAV based on the second-order consensus algorithm and formations gradually form during the flight. Based on the second-order consensus algorithm, the UAV group can achieve formation maintenance easily and the Laplacian matrix used in the algorithm is symmetric for an undirected graph. In the process of obstacle avoidance, the improved artificial potential field method can solve the jitter problem that the traditional artificial potential field method causes for the UAV and avoids violent jitter. Finally, simulation experiments of two scenarios were designed to verify the collision avoidance effect and formation retention effect of static obstacles and dynamic obstacles while the two UAV groups fly in opposite symmetry in the dynamic obstacle scenario. The experimental results demonstrate the effectiveness of the proposed method.