In modern industrial systems, bearing failures account for 30–40% of industrial machinery faults. Traditional convolutional neural network suffers from gradient vanishing and overfitting, resulting in a poor diagnostic accuracy. To address the issues, a new bearing fault diagnosis approach was proposed based on an improved AlexNet neural network combined with transfer learning. After decomposition and noise-reduction, reconstructed vibration signals were transformed into 2D images, then input into the improved AlexNet for training and follow-up transfer learning. Program auto-tuning and image-enhancing techniques were employed to increase the diagnostic accuracy in this study. The approach was verified with the datasets from Case Western Reserve University (CWRU), Jiangnan University (JNU), and the Association for Mechanical Failure Prevention Technology (MFPT). The results showed that the diagnostic accuracies by normal learning were more than 97% for CWRU and JNU datasets, and 100% for MFPT dataset. After transfer learning, the accuracies all reached above 99.5%. The proposed approach was demonstrated to be able to effectively diagnose the bearing faults.