Defects detection technology is essential for monitoring and hence maintaining the product quality of additive manufacturing (AM) processes; however, traditional detection methods based on single sensor have great limitations such as low accuracy and scarce information. In this study, a multi-sensor defect detection system (MSDDS) was proposed and developed for defect detection with the fusion of visible, infrared, and polarization detection information. The assessment criteria for imaging quality of the MSDDS have been optimized and evaluated. Meanwhile, the feasibility of processing and assembly of each sensor module has been demonstrated with tolerance sensitivity and the Monte Carlo analysis. Moreover, multi-sensor image fusion processing, super-resolution reconstruction, and feature extraction of defects are applied. Simulation and experimental studies indicate that the developed MSDDS can obtain high contrast and clear key information, and high-quality detected images of AM defects such as cracking, scratches, and porosity can be effectively extracted. The research provides a helpful and potential solution for defect detection and processing parameter optimization in AM processes such as Selective Laser Melting.