External defects of kiwifruit seriously affect its added commercialization. To address the existing problems, kiwifruit external defects detection has a few methods for detecting multi-category defects and weak adaptability to complex images. In this study, we proposed ResNet combined with CBAM for the automatic detection of external defects in kiwifruit. The experiment first built an acquisition device to obtain high-quality images. The optimal fusion scheme of ResNet and CBAM was investigated, the network training parameters were optimized, and Adam was used to accelerate the convergence speed of the model. It was found that the average recognition accuracy of ResNet34 + CBAM for kiwifruit was 99.6%, and all evaluation metrics were greater than 99%. Meanwhile, the experiment selected AlexNet, VGG16, InceptionV3, ResNet34, and ResNet34 + CBAM for comparison. The results showed that the recognition accuracy of ResNet34 + CBAM was 7.9%, 12.7%, 11.8%, and 4.3% higher than that of AlexNet, VGG16, InceptionV3, and ResNet34, respectively. Therefore, it can be concluded that ResNet34 + CBAM has the advantages of high recognition accuracy and good stability for kiwifruit external defect sample detection. It provides a technical guarantee for online detection and sorting of kiwifruit and other fruit defects.