Black phosphorus (BP) is a layered orthorhombic crystal with uniquely arranged atoms forming a crumpled honeycomb lattice. This special atomic arrangement gives BP unique optical anisotropy, which is expected to be widely used in polarized optics. However, conventional image analysis used to study its anisotropy is complex and inefficient. This paper proposed a machine-learning-based approach to conveniently identify black phosphorus's optical anisotropy features. Red−green−blue (RGB) values were extracted from regions of interest (ROI) with a consistent thickness by the detection algorithm, and then the data were processed to obtain a sample eigenvalue data set. Variations in the RGB values of the optical image directly reflect changes in the ability of black phosphorus to reflect polarized light. RGB was converted to grayscale, and it was found that they both change periodically with the rotation angle. Subsequently, redundant data were eliminated by meticulously assessing feature importance, reducing generalization errors. The performance of the models was evaluated in terms of accuracy, recall, F1_Score, and area under the receiver operating characteristic curve (AUC-ROC), all of which were found to be consistently above 0.9. Machine learning algorithmic models can accurately classify BP images with different rotation angles to identify the optical anisotropy features of BP. Machine learning algorithms can automatically learn from the data and improve the algorithms, bolstering problem-solving efficiency and precision. This minimizes human and material resource waste from experimental errors, fostering interdisciplinary synergy between materials science and artificial intelligence.