Cell tracking using deep neural networks with multi-task learning

Abstract

Cell tracking plays crucial role in biomedical and computer vision areas. As cells generally have frequent deformation activities and small sizes in microscope image, tracking the non-rigid and non-significant cells is quite difficult in practice. Traditional visual tracking methods have good performances on tracking rigid and significant visual objects, however, they are not suitable for cell tracking problem. In this paper, a novel cell tracking method is proposed by using Convolutional Neural Networks (CNNs) as well as Multi-Task Learning (MTL) techniques. The CNNs learn robust cell features and MTL improves the generalization performance of the tracking. The proposed cell tracking method consists of a particle filter motion model, a multi-task learning observation model, and an optimized model update strategy. In the training procedure, the cell tracking is divided into an online tracking task and an accompanying classification task using the MTL technique. The observation model is trained by building a CNN to learn robust cell features. The tracking procedure is started by assigning the cell position in the first frame of a microscope image sequence. Then, the particle filter model is applied to produce a set of candidate bounding boxes in the subsequent frames. The trained observation model provides the confidence probabilities corresponding to all of the candidates and selects the candidate with the highest probability as the final prediction. Finally, an optimized model update strategy is proposed to enable the multitask observation model for the variation of the tracked cell over the entire tracking procedure. The performance and robustness of the proposed method is analyzed by comparing with other commonly-used methods. Experimental results demonstrate that the proposed method has good performance to the cell tracking problem.

Joint Multi-frame Detection and Segmentation for Multi-cell Tracking

“…Multi-cell tracking in image sequence is valuable for stem cell research, tissue engineering, drug discovery and proteomics [1]. Researchers can construct cell lineage trees and analyze cell morphology based on cell tracking results [2].…”

Deep Learning‐Based Single‐Cell Optical Image Studies

“…Ramesh et al proposed a multitask‐based CNN to detect and segment cells in microscopic images . He et al proposed a new cell tracking method based on CNNs and multitask learning techniques, they proved that MTL improves the generalization performance of tracking .…”

“…Multiple tasks such as classification, segmentation and tracking can be performed in parallel through multitask learning in single-cell optical image analysis, which cannot only enhance the implementation of multitask but also improve the generalization ability of the model. (33).…”

EllipTrack: A Global-Local Cell-Tracking Pipeline for 2D Fluorescence Time-Lapse Microscopy

“…Finally, in Signal Extraction, signals are extracted from the regions of interest in each cell to gain biological insights. During the last decades, enormous effort has been made to improve the accuracy of each step (He et al, 2017;Hernandez et al, 2018;Maška et al, 2014;Moen et al, 2019;Ulman et al, 2017). Some cell trackers also attempt to perform Segmentation and Track Linking jointly, such that the properties of cell tracks from previous frames can be used as prior information to track later frames (Amat et al, 2014;Arbelle et al, 2018;Cappell et al, 2016).…”