Bearing fault diagnosis is a critical component of the mechanical equipment monitoring system. In the complex and harsh environment in which bearings operate, the fault diagnosis approach of multi-source information fusion can extract fault features more stably and extensively than the traditional single-source fault diagnosis method. However, most existing multi-source fusion methods are in infancy, and there are a number of pressing issues to address, such as subjective elements having a significant impact, excessive data redundancy, fuzzy multi-source signal fusion strategy, and insufficient accuracy. As a result, a new multi-source fusion fault diagnosis method is proposed in this paper. First, the residual pyramid algorithm is utilized to fuse the acoustic and vibration signals of multiple spatial positions respectively and then form two fused acoustic and vibration signals. Second, two improved 2D-CNN are used to extract the fault features contained in the above two signals separately to form a multi-source fault feature set. Third, an AdaBoost algorithm with a dynamic deletion mechanism is designed to fuse multi-source fault feature sets and produce the fault diagnosis findings. Finally, six different experimental data sets are used to test the performance of the model. The results reveal that the model has better generalization, higher and more stable fault diagnostic accuracy, and stronger anti-interference capacity.