Due to the long-term service through wheel-rail rolling contact, the train wheelset tread will inevitably suffer from different types of defects, such as wear, cracks, and scratches. The effective detection of wheelset tread defects can provide critical support for the operation and maintenance of trains. In this paper, a new method based on a local inference constraint network is proposed to detect wheelset tread defects, and the main purpose is to address the issue of insufficient feature spaces caused by small samples. First, a generative adversarial network is applied to generate diverse samples with semantic consistency. An attention mechanism module is introduced into the feature extraction network to increase the importance of defect features. Then, the residual spine network for local input decisions is constructed to establish an association between sample features and defect types. Furthermore, the network’s activation function is improved to obtain higher learning speed and accuracy with fewer parameters. Finally, the validity and feasibility of the proposed method are verified using experimental data.