This work aims to construct an artificial neural network (ANN) ant colony algorithm (ACA)-based fine recognition system for plant factory seedling phenotypes. To address the problems of complexity and high delay of the plant recognition system in plant factories, first, multiple cameras at different positions are employed to collect images of seedlings and construct 3D images. Then, the mask region convolutional neural networks (MRCNN) algorithm is adopted to analyze plant phenotypes. Finally, the optimized ACA is employed to optimize the process timing in the plant factory, thereby constructing a plant factory seedling phenotype fine identification system via ANN combined with ACA. Moreover, the model performance is analyzed. The results show that plants have four stages of phenotypes, namely, the germination stage, seedling stage, rosette stage, and heading stage. The accuracy of the germination stage reaches 97.01%, and the required test time is 5.64 s. Additionally, the optimization accuracy of the process timing sequence of the proposed model algorithm is maintained at 90.26%, and the delay and energy consumption are stabilized at 20.17 ms and 17.71, respectively, when the data volume is 6000 Mb. However, the problem of image acquisition occlusion in the process of 3D image construction still needs further study. Therefore, the constructed ANN-ACA-based fine recognition system for plant seedling phenotypes can optimize the process timing in a more real-time and lower energy consumption way and provide a reference for the integrated progression of unmanned intelligent recognition systems and complete sets of equipment for plant plants in the later stage.