Defect detection of lithium batteries is a crucial step in lithium battery production. However, traditional detection methods mainly rely on the human eyes to observe the bottom defects of lithium battery products, which have low detection accuracy and slow detection speed. To solve this practical problem, an improved YOLOv5s model is proposed in this paper. Firstly, a new layer of the network output layer is added to improve the detection effect of small defects. Secondly, to extract important information in the feature maps, the Convolutional Block Attention Module (CBAM) attention mechanism is added to the YOLOv5s model. Finally, a new position loss function is used to improve the accuracy of the position prediction of the model. The experimental results indicate that the improved YOLOv5s model can accurately and quickly detect three types of defects on the bottom surface of lithium batteries. Specifically, the loss and mean average precision (mAP) of the improved YOLOv5s model are 0.03394 and 67.5% respectively. Compared with the traditional YOLOv5s model, the loss of the improved YOLOv5s model is reduced by 31%. As well as, the mAP of the improved YOLOv5s model is increased by 4.3% on the lithium battery defect dataset. Compared with the YOLOv3, YOLOv3-spp, RetinaNet and YOLOv4, the mAP of the improved YOLOv5s model increased by 5.4%, 0.7%, 11.9% and 3.7% respectively. Compared with other improved YOLOv5 algorithms used in various fields, the mAP of the proposed model on the lithium battery dataset is the highest. The detection speed of the improved YOLOv5s model reaches 111 frames per second (fps), which can meet the real-time detection requirements. The improved YOLOv5s model has board application prospects in the industrial production of lithium batteries.