To improve the accuracy of museum cultural relic image recognition, the DenseNet and ResNet are selected as the backbone neural networks for detection and recognition. In view of the small target problem in cultural relics, the feature pyramid is introduced in this paper to improve the DenseNet method. The accuracy of target detection is improved through multiscale feature extraction and fusion. At the same time, aiming the problem of weak robustness and feature extraction of cultural relic images, the attention mechanism is proposed to improve ResNet. Therefore, this network can pay attention to the key of feature areas in the image. Finally, the aforementioned methods are verified by experiments. The results show that compared with the YOLOv3 and other algorithms, the accuracy of the improved ResNet proposed in this experiment is above 90%. Furthermore, the number of missed and erroneous detection is the lowest, which are 171 and 134, respectively. The identified mAP indicator accuracy can reach 86%, which also exceeds SVD-Net and DenseNet. It can be seen that the constructed method can effectively detect and recognize the museum cultural relic images.