Wheels are one of the most important testing components in rail transport that play a significant role in the safety of train, and thence, research on wheel defect detection is of great significance. In this article, a method using image processing techniques and artificial neural network techniques is proposed for the purpose of recognizing defects in ultrasound B-scan image. A noise reduction and filtering algorithm and a feature extraction algorithm are proposed to simplify identification steps and improve the accuracy of the later recognition. Then, two back propagation neural networks with two hidden layers are built respectively for two classification steps. One is to identify noise, while the other is to identify the echo of real defects. Finally, a 93 % recognition rate is achieved by using the algorithm proposed in this article. The result shows that appropriate feature extraction algorithms and artificial neural network techniques are efficient and reliable in defects recognition of ultrasound B-scan images.