Identifying the grade of Gastrodia elata in the market has low efficiency and accuracy. To address this issue, an I-YOLOX object detection algorithm based on deep learning and computer vision is proposed in this paper. First, six types of Gastrodia elata images of different grades in the Gastrodia elata planting cooperative were collected for image enhancement and labeling as the model training dataset. Second, to improve feature information extraction, an ECA attention mechanism module was inserted between the backbone network CSPDarknet and the neck enhancement feature extraction network FPN in the YOLOX model. Then, the impact of the attention mechanism and application position on model improvement was investigated. Third, the 3 × 3 convolution in the neck enhancement feature extraction network FPN and the head network was replaced by depthwise separable convolution (DS Conv) to reduce the model size and computation amount. Finally, the EIoU loss function was used to predict boundary frame regression at the output prediction end to improve the convergence speed of the model. The experimental results indicated that compared with the original YOLOX model, the mean average precision of the improved I-YOLOX network model was increased by 4.86% (97.83%), the model computation was reduced by 5.422 M (reaching 3.518 M), the model size was reduced by 20.6 MB (reaching 13.7 MB), and the image frames detected per second increased by 3 (reaching 69). Compared with other target detection algorithms, the improved model outperformed Faster R-CNN, SSD-VGG, YOLOv3s, YOLOv4s, YOLOv5s, and YOLOv7 algorithms in terms of mean average precision, model size, computation amount, and frames per second. The lightweight model improved the detection accuracy and speed of different grades of Gastrodia elata and provided a theoretical basis for the development of online identification systems of different grades of Gastrodia elata in practical production.