Automated Video Behavior Recognition of Pigs Using Two-Stream Convolutional Networks

Zhang, Kaifeng; Li, Dan; Huang, Jichao; Chen, Yifei

doi:10.3390/s20041085

Cited by 40 publications

(22 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Body dimension (G), weight (G), movement (G, S), farrowing alarms (S) [32,[38][39][40][41][42]45,49,62,64,68,[88][89][90][91]93,96] a A: abattoir, G: growing pigs, P: piglets, S: sows. Stress vocalisation (G, P), object manipulation (G), defence cascade response (G), pig face recognition (S) [33,57,66,74,76,[79][80][81]87] a A: abattoir, G: growing pigs, P: piglets, S: sows.…”

Section: Good Housing 42mentioning

confidence: 99%

Information Technologies for Welfare Monitoring in Pigs and Their Relation to Welfare Quality®

2021

View full text Add to dashboard Cite

The assessment of animal welfare on-farm is important to ensure that current welfare standards are followed. The current manual assessment proposed by Welfare Quality® (WQ), although being an essential tool, is only a point-estimate in time, is very time consuming to perform, only evaluates a subset of the animals, and is performed by the subjective human. Automation of the assessment through information technologies (ITs) could provide a continuous objective assessment in real-time on all animals. The aim of the current systematic review was to identify ITs developed for welfare monitoring within the pig production chain, evaluate the ITs developmental stage and evaluate how these ITs can be related to the WQ assessment protocol. The systematic literature search identified 101 publications investigating the development of ITs for welfare monitoring within the pig production chain. The systematic literature analysis revealed that the research field is still young with 97% being published within the last 20 years, and still growing with 63% being published between 2016 and mid-2020. In addition, most focus is still on the development of ITs (sensors) for the extraction and analysis of variables related to pig welfare; this being the first step in the development of a precision livestock farming system for welfare monitoring. The majority of the studies have used sensor technologies detached from the animals such as cameras and microphones, and most investigated animal biomarkers over environmental biomarkers with a clear focus on behavioural biomarkers over physiological biomarkers. ITs intended for many different welfare issues have been studied, although a high number of publications did not specify a welfare issue and instead studied a general biomarker such as activity, feeding behaviour and drinking behaviour. The ‘good feeding’ principle of the WQ assessment protocol was the best represented with ITs for real-time on-farm welfare assessment, while for the other principles only few of the included WQ measures are so far covered. No ITs have yet been developed for the ‘Comfort around resting’ and the ‘Good human-animal relationship’ criteria. Thus, the potential to develop ITs for welfare assessment within the pig production is high and much work is still needed to end up with a remote solution for welfare assessment on-farm and in real-time.

show abstract

Section: Good Housing 42mentioning

confidence: 99%

Information Technologies for Welfare Monitoring in Pigs and Their Relation to Welfare Quality®

2021

View full text Add to dashboard Cite

show abstract

“…General solutions are to manually remove invalid files and retain informative data for the development. Examples of invalid files include blurred images, images without targets or only with parts of targets, and images without diverse changes [ 11 , 51 , 80 , 90 , 107 ]. Some image processing algorithms were available to compare the differences between adjacent frames or between background frames and frames to be tested and capable of automatically ruling out unnecessary files.…”

Section: Preparationsmentioning

confidence: 99%

“…One strategy to improve efficiency is to crop an image into regions of interest before processing. The cropping can be based on a whole body of an animal [ 80 , 117 , 118 ], parts (e.g., face, trunk) of an animal [ 55 , 107 ], or areas around facilities (e.g., enrichment, feeder, and drinker) [ 86 , 114 , 119 ]. As for some large images, cropping them into small and regular pieces of images can reduce computational resources and improve processing speed [ 56 , 64 , 106 , 120 ].…”

Section: Preparationsmentioning

confidence: 99%

“…These are useful architectures to achieve performance of interest for target tracking. [75] SlowFast network [225] Low and high frame rates, slow and high pathways, lightweight network GitHub [226] (PyTorch) [47] Two-stream CNN [221] Complementary information on appearance, motion between frames, GitHub [227] (Python) [80] (Inception, ResNet, VGG, and Xception) with LSTM [222] Recurrent convolution, CNN, doubly deep in spatial and temporal layers, LSTM GitHub [228] (PyTorch) GitHub [229] (PredNet) [86,87,119], etc.…”

Section: Architectures For Trackingmentioning

confidence: 99%

See 1 more Smart Citation

Practices and Applications of Convolutional Neural Network-Based Computer Vision Systems in Animal Farming: A Review

Huang

Chen

et al. 2021

Sensors

102

View full text Add to dashboard Cite

Convolutional neural network (CNN)-based computer vision systems have been increasingly applied in animal farming to improve animal management, but current knowledge, practices, limitations, and solutions of the applications remain to be expanded and explored. The objective of this study is to systematically review applications of CNN-based computer vision systems on animal farming in terms of the five deep learning computer vision tasks: image classification, object detection, semantic/instance segmentation, pose estimation, and tracking. Cattle, sheep/goats, pigs, and poultry were the major farm animal species of concern. In this research, preparations for system development, including camera settings, inclusion of variations for data recordings, choices of graphics processing units, image preprocessing, and data labeling were summarized. CNN architectures were reviewed based on the computer vision tasks in animal farming. Strategies of algorithm development included distribution of development data, data augmentation, hyperparameter tuning, and selection of evaluation metrics. Judgment of model performance and performance based on architectures were discussed. Besides practices in optimizing CNN-based computer vision systems, system applications were also organized based on year, country, animal species, and purposes. Finally, recommendations on future research were provided to develop and improve CNN-based computer vision systems for improved welfare, environment, engineering, genetics, and management of farm animals.

show abstract

“…Zhang et al proposed two-stream pigs behaviors recognition based on RestNet101, which classify feeding, lying, walking, scratching, and mounting behaviors with a global accuracy of 98.99% [50]. Li et al used Mask R-CNN, an extension of Faster R-CNN to segment pigs followed by kernel-extreme learning machine to detect mounting behavior with an accuracy of 91.47%, a sensitivity of 95.2%, and a specificity of 88.34% [51].…”

Section: Pigsmentioning

confidence: 99%

Farm Animals’ Behaviors and Welfare Analysis with IA Algorithms: A Review

Debauche¹,

Elmoulat²,

Mahmoudi³

et al. 2021

RIA

View full text Add to dashboard Cite

Numerous bibliographic reviews related to the use of AI for the behavioral detection of farm animals exist, but they only focus on a particular type of animal. We believe that some techniques were used for some animals that could also be used for other types of animals. The application and comparison of these techniques between animal species are rarely done. In this paper, we propose a review of machine learning approaches used for the detection of farm animals’ behaviors such as lameness, grazing, rumination, and so on. The originality of this paper is matched classification in the midst of sensors and algorithms used for each animal category. First, we highlight the most implemented approaches for different categories of animals (cows, sheep, goats, pigs, horses, and chickens) to inspire researchers interested to conduct investigation and employ the methods we have evaluated and the results we have obtained in this study. Second, we describe the current trends in terms of technological development and new paradigms that will impact the AI research. Finally, we critically analyze what is done and we draw new pathways of research to advance our understanding of animal’s behaviors.

show abstract

Automated Video Behavior Recognition of Pigs Using Two-Stream Convolutional Networks

Cited by 40 publications

References 31 publications

Information Technologies for Welfare Monitoring in Pigs and Their Relation to Welfare Quality®

Information Technologies for Welfare Monitoring in Pigs and Their Relation to Welfare Quality®

Practices and Applications of Convolutional Neural Network-Based Computer Vision Systems in Animal Farming: A Review

Farm Animals’ Behaviors and Welfare Analysis with IA Algorithms: A Review

Contact Info

Product

Resources

About