A Robust Real-Time Detecting and Tracking Framework for Multiple Kinds of Unmarked Object

Abstract

A rodent real-time tracking framework is proposed to automatically detect and track multi-objects in real time and output the coordinates of each object, which combines deep learning (YOLO v3: You Only Look Once, v3), the Kalman Filter, improved Hungarian algorithm, and the nine-point position correction algorithm. A model of a Rat-YOLO is trained in our experiment. The Kalman Filter model is established in an acceleration model to predict the position of the rat in the next frame. The predicted data is used to fill the losing position of rats if the Rat-YOLO doesn't work in the current frame, and to associate the ID between the last frame and current frame. The Hungarian assigned algorithm is used to show the relationship between the objects of the last frame and the objects of the current frame and match the ID of the objects. The nine-point position correction algorithm is presented to adjust the correctness of the Rat-YOLO result and the predicted results. As the training of deep learning needs more datasets than our experiment, and it is time-consuming to process manual marking, automatic software for generating labeled datasets is proposed under a fixed scene and the labeled datasets are manually verified in term of their correctness. Besides this, in an off-line experiment, a mask is presented to remove the highlight. In this experiment, we select the 500 frames of the data as the training datasets and label these images with the automatic label generating software. A video (of 2892 frames) is tested by the trained Rat model and the accuracy of detecting all the three rats is around 72.545%, however, the Rat-YOLO combining the Kalman Filter and nine-point position correction arithmetic improved the accuracy to 95.194%.Sensors 2020, 20, 2 2 of 17 to children with autism [4]. As the social behavior is more complex than we can image, some works focus on the interaction between two rats [5,6], and real-time target tracking throughout the process provides a more comprehensive analysis of animal neural activity in studying animal neural activity. In the past survey of social behavior, the focus of research has been the interaction between the two rats in a cage [7,8]. Therefore, the research of social behavior with more than two rats is a good research direction to study SI in the future.The development of machine vision has been ongoing rapidly along with the development of deep learning in recent years. The current detection method of rats is based on the traditional image processing methods, such as background subtraction [9,10], but by this method, the background is generally set as simple, and there is a great contrast between the detected object and background in the histogram of the image. However, some social behavior requires experiments to be in different scenes, and the experimental background is dynamic rather than static; thus, the method make it more possible to recognize fault objects in the condition, Furthermore, using more complex arithmetic solves this problem which may consume more time. In c...

PMotion: an advanced markerless pose estimation approach based on novel deep learning framework used to reveal neurobehavior

“…The data enhancement procedures used in previous neural networks for pose recognition were rotation [28,29], flip [17], and scaling [28]. Although the background is fixed under laboratory conditions, the contrast between the background and foreground is insufficient under certain conditions, and the interference by light is stronger [19], the animals in motion have variable speeds and may have blurred images under conditions where the camera frame rate is insufficient. Thus, for blurred images, an augmenter is used with Gaussian kernels and adjustment of image contrast by scaling pixel values.…”

“…Convolutional neural network provides a method of feature extraction of two-dimensional (2D) images [18,19] and is proven to be effective for image segment analysis [20][21][22]. With improvements in the framework, self-attention mechanisms have been added to the convolutional network [21,[23][24][25], which strengthen the local and global features of an image segment.…”

“…The deeplabcut [11], deeplabcut-live [34], deepposekit [17] and LEAP [9] are widely used in behavioral science research for animal pose estimation [35,36]. Despite the innovations in target recognition algorithms, errors in the first stage of object detection [19] make pose estimation techniques unreliable, unpractical, and inaccurate, creating a significant drawback to their use.…”

Multiple Object Tracking in Deep Learning Approaches: A Survey

“…This paper consists of two modules: a behavior recognition module and a state evolution module. Lv et al proposed a depthwise separable convolution neural network (DS-CNN) [121], which included pointwise convolution(P-Conv2D) and depthwise convolution (D-Conv2D). Xu et al proposed the Group Feature Selection Method for Discriminative Correlation Filters (GFS-DCF) [122] to select group features at the channel and spatial dimensions.…”

Deep learning-based visual control assistant for assembly in Industry 4.0

“…YOLO is an architecture specialized in the detection and tracking of multiple objects in real time, generating coordinates for each object. To achieve this high-speed detection, the precision is reduced, although the method retains very high levels of accuracy (Aggarwal, 2018;Lv et al, 2020). This capacity can be used to perform actions such as autonomous driving without specialized sensors (Redmon et al, 2016).…”