Yarn hairiness affects yarn and fabric quality. The existing hairiness detection methods cannot discriminate crossover fibers or hairs. In order to accurately separate and detect crossover fibers, an algorithm of separating crossover hairs is proposed. By obtaining refined hair skeletons after image pretreatment, the positions of fiber intersection point were determined, and a hair information table according to the characteristics of the hair cross-point was constructed. After classifying each hair branch skeleton and screening out the hair common skeleton, the branch hair matching table by using the two end points of the true common hair skeleton adjacent to the hair branch was constructed. Through pairing the same cross-hair branch with the principle of the closest slope at the adjacent cross-end, each complete cross-hair skeleton was defined for hair count in a field of view. The detection results show that compared with the existing photoelectric hairiness detection instrument, the algorithm can realize the crossover hair separation, and calculate the length of complete crossover hairs and curved hairs with high accuracy. On average, the developed algorithm measures hair length about 11.1% longer than the manually measured results, while commercial apparatus would report hair length 62.5% shorter than the actual hair length.