To solve the chip location recognition problem, this paper proposes a lightweight E-YOLOv5 based chip detection algorithm based on the You Only Look Once version 5 (YOLOv5s) algorithm. For the problem of the difficult distinction between chip detection points and light spots, a simulated exposure algorithm is used to process part of the training set images to enhance model robustness; the existing model network is complex, and EfficientNet, a lightweight feature extraction network, is introduced to reduce the model size; for the problem of imprecise model recognition due to small detection points, Selective Kernel Neural Network (SKNet) module is introduced into EfficientNet is introduced to enhance the feature extraction ability of the model and improve the training efficiency, and Efficient Intersection over Union Loss (EIoU_Loss) is used as the loss function to reduce the false recognition rate. Experiments show that the algorithm in this paper improves by 3.85% and 3.92% in precision, recall rate, 28.89% in loss value, nearly 20% in model size and training time, and 46.67% in image processing speed on CPU compared with YOLOv5s. The experimental results show that the proposed algorithm outperforms other algorithms and is able to distinguish and identify chip locations precisely and stably.