Huanglongbing (HLB) is a highly contagious and devastating citrus disease that causes huge economic losses to the citrus industry. Because it cannot be cured, timely detection of the HLB infection status of plants and removal of diseased trees are effective ways to reduce losses. However, complex HLB symptoms, such as single HLB-symptomatic or zinc deficiency + HLB-positive, cannot be identified by a single reflection imaging method at present. In this study, a vision system with an integrated reflection–transmission image acquisition module, human–computer interaction module, and power supply module was developed for rapid HLB detection in the field. In reflection imaging mode, 660 nm polarized light was used as the illumination source to enhance the contrast of the HLB symptoms in the images based on the differences in the absorption of narrow-band light by the components within the leaves. In transmission imaging mode, polarization images were obtained in four directions, and the polarization angle images were calculated using the Stokes vector to detect the optical activity of starch. A step-by-step classification model with four steps was used for the identification of six classes of samples (healthy, HLB-symptomatic, zinc deficiency, zinc deficiency + HLB-positive, magnesium deficiency, and boron deficiency). The results showed that the model had an accuracy of 96.92% for the full category of samples and 98.08% for the identification of multiple types of HLB (HLB-symptomatic and zinc deficiency + HLB-positive). In addition, the classification model had good recognition of zinc deficiency and zinc deficiency + HLB-positive samples, at 92.86%.