Effectively predicting the remaining using life (RUL) of the rolling bearings can ensure the reliability and safety, which can make full use of their life, minimize machine downtime, and reduce the operation and maintenance cost of enterprises. To solve the problems of data distribution discrepancy caused by different working conditions and the collected signals containing a lot of useless information and noise, a novel cross-domain adaption network (CDAN) is proposed in this study. Firstly, a novel feature extractor, Se-Sk-DenseNet, is developed to extract useful critical features from the input data and remove the ineffective features by embedding Se and Sk attention blocks; Besides, a new objective loss function consist of the RUL loss, the MK-MMD loss, the contrastive loss, and the KL divergence loss, is proposed to solve the problem of data distribution shift; Finally, the effectiveness and superiority of CDAN are proved on the PHM2012 bearings dataset. The results demonstrate that CDAN can extract deep critical features and achieve the high cross-domain RUL prediction accuracy under different working conditions.