Angelica sinensis (Oliv.) Diels, a member of the Umbelliferae family, is commonly known as Danggui (Angelica sinensis, AS). AS has the functions of blood tonic, menstrual pain relief, and laxatives. Accurate classification of AS grades is crucial for efficient market management and consumer health. The commonly used method to classify AS grades depends on the evaluator’s observation and experience. However, this method has issues such as unquantifiable parameters and inconsistent identification results among different evaluators, resulting in a relatively chaotic classification of AS in the market. To address these issues, this study introduced a computer vision-based approach to intelligently grade AS. Images of AS at five grades were acquired, denoised, and segmented, followed by extraction of shape, color, and texture features. Thirteen feature parameters were selected based on difference and correlation analysis, including tail area, whole body area, head diameter, G average, B average, R variances, G variances, B variances, R skewness, G skewness, B skewness, S average, and V average, which exhibited significant differences and correlated with grades. These parameters were then used to train and test both the traditional back propagation neural network (BPNN) and the BPNN model improved with a growing optimizer (GOBPNN). Results showed that the GOBPNN model achieved significantly higher average testing precision, recall, F-score, and accuracy (97.1%, 95.9%, 96.5%, and 95.0%, respectively) compared to the BPNN model. The method combining machine vision technology with GOBPNN enabled efficient, objective, rapid, non-destructive, and cost effective AS grading.