Recently, Mueller matrix polarimetry has received increasing attention in the field of biophotonics, because of its great potential for non-invasive, label-free detection of microstructural and optical properties of biomedical samples. In this study, we propose a method for automatic identification and quantitative evaluation of skin hair follicle structure based on Mueller matrix polarimetry combined with the K-means clustering machine learning algorithm. First, we use the transmission Mueller matrix microscope to measure the rat skin tissue sections with hair follicles. Then we derive the Mueller matrix transformation (MMT) parameters images to reveal the characteristics of the birefringent skin hair follicle structure. By taking the MMT parameters images as the identification objects, we adopt the K-means clustering algorithm to segment them and carry out image denoising processing to achieve the automatic detection of the hair follicle structure. Finally, to identify the hair follicle structure quantitatively and accurately, we conduct a comprehensive evaluation of five indexes including quantity, area, position, long axis, and short axis of the recognized regions. The results show that the method presented in this study can realize the automatic identification and quantitative evaluation of skin hair follicle structures, having great potential for the detection and clinical diagnosis of skin structures.