Currently, most fault diagnosis methods based on domain adaptation reduce the distribution discrepancy between the source and target domains by aligning their high-dimensional features, but do not consider the impact of the source domain decision boundary on the distribution of the target domain, which leads to slow convergence and poor generalization. Aiming at the above-mentioned issues, this paper proposes a double-aligned domain adaptation deep transfer model (MSADA) based on multi-structure fusion and multi-attention mechanisms. Firstly, a multi-structure fusion network with Inception module is constructed to increase the information capacity in the extracted features and enhance the representation of deep features. Then, the multi-attention mechanism is embedded into different convolutional channels separately to learn globally and locally important information and reduce the influence of the source domain decision boundary on the target domain distribution. Finally, the multi-kernel maximum mean discrepancy (MK-MMD) and KL divergence are combined as a new double-aligned distribution discrepancy metric to align the samples and probability distributions of the source domain and the target domain, respectively. At the same time, a dynamic adaptive factor is designed to adjust the contribution of the two types of distributions, thus effectively improving the training efficiency and the robustness of the model. Through the validation analysis of two rolling bearing dataset cases, the proposed MSADA has better cross-domain diagnostic performance than other domain adaptation methods.