With the widely application and promotion of Unmanned aerial vehicle (UAV) technology, UAV swarms are widely used in military and civilian fields by taking advantage of group collaboration. It has huge economic and national defense value. The safety and reliability requirements of the UAV swarm system are extremely strict, and real-time fault detection and diagnosis is one of its important supporting technologies. In this paper, a fault diagnosis method based on statistical model and improved broad learning system (BLS) is proposed. The behavior characteristics of the UAV swarm system under normal and different failure modes are characterized by multivariate data statistical analysis, and the improved BLS model is adopted to achieve accurate and fast fault diagnosis. And a high-fidelity simulation verification platform is developed to verify the rationality and effectiveness of the proposed method.