The automatic classification of surface defects is significant to the steel strip inspection system. However, influenced by the lack of samples, it is difficult to improve the classification accuracy. Herein, a novel dual‐stream neural network is proposed, which is composed by two streams of sample generation and classification training. Subsequently, numerous defect samples are generated for the classifier pretraining, and the real steel strip surface defects are classified by the transfer learning method. The experiment results demonstrate that the dual‐stream neural network has the highest classification accuracy of 99.70% among the state‐of‐the‐art classifiers, and the classification accuracy increasesd by 9.67% through generating plentiful similar images for pretraining. Moreover, the convolutional neural network (CNN) can improve the quality of generated defect images. When the number of epochs reaches 2000, the generated images are closest to the real labeled images. Finally, the cross‐validation proves that the dual‐stream neural network has excellent stability and robustness, without over‐fitting problem in the training process.