In the process of grain acquisition, the unsound kernel of wheat is detected mainly by manual detection, and the method of detection by computer vision is still in the experimental stage. Aiming at the problems such as expensive equipment for image acquisition, difficulty in adhesion segmentation, and low recognition efficiency in detection, this paper takes six kinds of wheat as objects, namely sound kernel, broken kernel, sprouted kernel, injured kernel, moldy kernel and spotted kernel, builds a wheat image acquisition platform, and establishes a two-kernels adhesion wheat segmentation algorithm based on concave-mask. Among the total 9988 wheat grains, the error rate accounts for 0.93%. By comparing the advantages and disadvantages of GoogleNet, DenseNet, IX-ResNet, Res2Net, exploring the improvement of depth, width, downsampling mode, convolution order, attention mechanism18, receptive field, and finally puts forward a wheat unsound kernel detection model based on Res24_D_CBAM_Atrous. The Macro avg values of Precision, Recall and F1 are 94%, 95% and 94%, respectively, which are increased by 3%-4% based on the original Res34 and the prediction time is reduced by 220s, which can meet the rapid and accurate evaluation of wheat appearance quality, and has important theoretical significance and practical application value for wheat market circulation.