Multicow pose estimation based on keypoint extraction

Abstract

Automatic estimation of the poses of dairy cows over a long period can provide relevant information regarding their status and well-being in precision farming. Due to appearance similarity, cow pose estimation is challenging. To monitor the health of dairy cows in actual farm environments, a multicow pose estimation algorithm was proposed in this study. First, a monitoring system was established at a dairy cow breeding site, and 175 surveillance videos of 10 different cows were used as raw data to construct object detection and pose estimation data sets. To achieve the detection of multiple cows, the You Only Look Once (YOLO)v4 model based on CSPDarkNet53 was built and fine-tuned to output the bounding box for further pose estimation. On the test set of 400 images including single and multiple cows throughout the whole day, the average precision (AP) reached 94.58%. Second, the keypoint heatmaps and part affinity field (PAF) were extracted to match the keypoints of the same cow based on the real-time multiperson 2D pose detection model. To verify the performance of the algorithm, 200 single-object images and 200 dual-object images with occlusions were tested under different light conditions. The test results showed that the AP of leg keypoints was the highest, reaching 91.6%, regardless of day or night and single cows or double cows. This was followed by the AP values of the back, neck and head, sequentially. The AP of single cow pose estimation was 85% during the day and 78.1% at night, compared to double cows with occlusion, for which the values were 74.3% and 71.6%, respectively. The keypoint detection rate decreased when the occlusion was severe. However, in actual cow breeding sites, cows are seldom strongly occluded. Finally, a pose classification network was built to estimate the three typical poses (standing, walking and lying) of cows based on the extracted cow skeleton in the bounding box, achieving precision of 91.67%, 92.97% and 99.23%, respectively. The results showed that the algorithm proposed in this study exhibited a relatively high detection rate. Therefore, the proposed method can provide a theoretical reference for animal pose estimation in large-scale precision livestock farming.

Applications of livestock monitoring devices and machine learning algorithms in animal production and reproduction: an overview

“…Health and behavior : many studies proposed AI and sensor-based alternatives to identify, monitor and control health-related features, such as early mastitis identification in dairy cows ( Wang et al, 2022 ), gait analysis and lameness detection ( Zhao et al, 2018 ; Nejati et al, 2023 ), respiratory pattern ( Wu et al, 2020 , 2023 ), diarrhea and respiratory disease in calves ( Ghaffari et al, 2022 ), body condition score ( Huang et al, 2019 ), body temperature measurement ( Giro et al, 2019 ; Barreto et al, 2022 ), and behavior monitoring ( Li et al, 2019 ; Watanabe et al, 2021 ; Gong et al, 2022 ).…”

GAM-YOLOv7-tiny and Soft-NMS-AlexNet: Improved lightweight sheep body object detection and pose estimation network

“…Wang K et al 4 proposed a foraging behavior recognition and classi cation algorithm based on feature extraction techniques and deep learning in 2021 using a representative acoustic dataset generated from typical weight sheep grazing on various grasses and subsequently regurgitating for experiments and eventually achieved an accuracy of 96.13%. In 2022, Caili Gong et al 5 constructed the You Only Look Once (YOLO) v4 model based on CSPDarkNet53 to achieve multiple cattle detection and ne-tuned it to output bounding boxes for further pose estimation. The results showed that their method achieved an average accuracy (AP) of 94.58% on a test set of 400 images throughout the day, with a relatively low detection speed of 8.06 f/s.…”

Real-Time Cattle Pose Estimation Based on Improved RTMPose

“…Testing on the mammalian public dataset, AP-10K demonstrated superior performance compared to lightweight networks, such as Lite-HRNet and HRformer-Tiny. In the same year, Gong et al [9] proposed a two-branch skeleton extraction network for multi-cattle pose estimation, conducting tests on single-target and dual-target images under varying lighting conditions, with singlecattle pose estimation achieving an accuracy of 85% during the day and 78.1% at night while dual-cattle pose estimation accuracy reached 74.3% and 71.6%. In 2023, Fan et al [10] designed CoordConv and DO-Conv, which were applied to the bottleneck and basic block of the HRNet network to improve cattle pose estimation performance.…”