Deep tracking in the wild: End-to-end tracking using recurrent neural networks

Dequaire, Julie; Ondrúška, Peter; Rao, Dushyant; Wang, Dominic Zeng; Posner, Ingmar

doi:10.1177/0278364917710543

Cited by 95 publications

(80 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Prediction of a time series using recurrence neural networks (RNN) has been primarily investigated in the context of natural language processing, [4][5][6][7][8] financial stock market prediction, 9,10 and computer vision problems including object recognition, 11,12 tracking, 13,14 and image caption. 15 Recently RNN has been rapidly extended to health-care applications and achieved a great success in electronic health records analysis, 16,17 disease progression analysis, 18,19 and analysis of tumor cell growth.…”

Section: Introductionmentioning

confidence: 99%

Toward predicting the evolution of lung tumors during radiotherapy observed on a longitudinal MR imaging study via a deep learning algorithm

Wang

Rimner

et al. 2019

Medical Physics

View full text Add to dashboard Cite

Purpose To predict the spatial and temporal trajectories of lung tumor during radiotherapy monitored under a longitudinal magnetic resonance imaging (MRI) study via a deep learning algorithm for facilitating adaptive radiotherapy (ART). Methods We monitored 10 lung cancer patients by acquiring weekly MRI‐T2w scans over a course of radiotherapy. Under an ART workflow, we developed a predictive neural network (P‐net) to predict the spatial distributions of tumors in the coming weeks utilizing images acquired earlier in the course. The three‐step P‐net consisted of a convolutional neural network to extract relevant features of the tumor and its environment, followed by a recurrence neural network constructed with gated recurrent units to analyze trajectories of tumor evolution in response to radiotherapy, and finally an attention model to weight the importance of weekly observations and produce the predictions. The performance of P‐net was measured with Dice and root mean square surface distance (RMSSD) between the algorithm‐predicted and experts‐contoured tumors under a leave‐one‐out scheme. Results Tumor shrinkage was 60% ± 27% (mean ± standard deviation) by the end of radiotherapy across nine patients. Using images from the first three weeks, P‐net predicted tumors on future weeks (4, 5, 6) with a Dice and RMSSD of (0.78 ± 0.22, 0.69 ± 0.24, 0.69 ± 0.26), and (2.1 ± 1.1 mm, 2.3 ± 0.8 mm, 2.6 ± 1.4 mm), respectively. Conclusion The proposed deep learning algorithm can capture and predict spatial and temporal patterns of tumor regression in a longitudinal imaging study. It closely follows the clinical workflow, and could facilitate the decision‐making of ART. A prospective study including more patients is warranted.

show abstract

Section: Introductionmentioning

confidence: 99%

Toward predicting the evolution of lung tumors during radiotherapy observed on a longitudinal MR imaging study via a deep learning algorithm

Wang

Rimner

et al. 2019

Medical Physics

View full text Add to dashboard Cite

show abstract

“…Many studies on multi-target tracking have been conducted in recent decades, which can be classified into two main categories. One category employs deep learning and computer vision techniques to track objects by real-time detection on images and videos [1], [2], where the bounding boxes of tracking targets can be obtained. The other category estimates the state distribution of tracking targets through Bayesian inference methods.…”

Section: Introductionmentioning

confidence: 99%

Generic Tracking and Probabilistic Prediction Framework and Its Application in Autonomous Driving

Zhan

et al. 2020

IEEE Trans. Intell. Transport. Syst.

View full text Add to dashboard Cite

Accurately tracking and predicting behaviors of surrounding objects are key prerequisites for intelligent systems such as autonomous vehicles to achieve safe and high-quality decision making and motion planning. However, there still remain challenges for multi-target tracking due to object number fluctuation and occlusion. To overcome these challenges, we propose a constrained mixture sequential Monte Carlo (CMSMC) method in which a mixture representation is incorporated in the estimated posterior distribution to maintain multi-modality. Multiple targets can be tracked simultaneously within a unified framework without explicit data association between observations and tracking targets. The framework can incorporate an arbitrary prediction model as the implicit proposal distribution of the CMSMC method. An example in this paper is a learning-based model for hierarchical time-series prediction, which consists of a behavior recognition module and a state evolution module. Both modules in the proposed model are generic and flexible so as to be applied to a class of time-series prediction problems where behaviors can be separated into different levels. Finally, the proposed framework is applied to a numerical case study as well as a task of on-road vehicle tracking, behavior recognition, and prediction in highway scenarios. Instead of only focusing on forecasting trajectory of a single entity, we jointly predict continuous motions for interactive entities simultaneously. The proposed approaches are evaluated from multiple aspects, which demonstrate great potential for intelligent vehicular systems and traffic surveillance systems.Index Terms-Trajectory prediction, multi-target tracking, sequential Monte Carlo, probabilistic graphical models, deep learning, bahavior recognition.

show abstract

“…In [20] an end-to-end trainable framework is proposed to predict un-occluded occupancy grid maps in an unsupervised manner, based on recurrent neural networks. The basic idea is to capture the dynamic evolution of the scene in a sequence of occupancy grid maps with a recurrent neural network.…”

Section: Introductionmentioning

confidence: 99%

“…We differentiate our work by not only providing a filtered occupancy grid map, but also velocity estimates for the occupied cells. Furthermore, the data in [20] is represented as grid maps with a size of 100 × 100, which is significant smaller than our data representation.…”

Section: Introductionmentioning

confidence: 99%

Long-Term Occupancy Grid Prediction Using Recurrent Neural Networks

Schreiber

Hoermann

Dietmayer

2019

2019 International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

In this work, we tackle the problem of modeling the vehicle environment as dynamic occupancy grid map in complex urban scenarios using recurrent neural networks. Dynamic occupancy grid maps represent the scene in a bird's eye view, where each grid cell contains the occupancy probability and the two dimensional velocity. As input data, our approach relies on measurement grid maps, which contain occupancy probabilities, generated with lidar measurements. Given this configuration, we propose a recurrent neural network architecture to predict a dynamic occupancy grid map, i.e. filtered occupancy and velocity of each cell, by using a sequence of measurement grid maps. Our network architecture contains convolutional long-short term memories in order to sequentially process the input, makes use of spatial context, and captures motion. In the evaluation, we quantify improvements in estimating the velocity of braking and turning vehicles compared to the state-of-the-art. Additionally, we demonstrate that our approach provides more consistent velocity estimates for dynamic objects, as well as, less erroneous velocity estimates in static area.

show abstract

Deep tracking in the wild: End-to-end tracking using recurrent neural networks

Cited by 95 publications

References 21 publications

Toward predicting the evolution of lung tumors during radiotherapy observed on a longitudinal MR imaging study via a deep learning algorithm

Toward predicting the evolution of lung tumors during radiotherapy observed on a longitudinal MR imaging study via a deep learning algorithm

Generic Tracking and Probabilistic Prediction Framework and Its Application in Autonomous Driving

Long-Term Occupancy Grid Prediction Using Recurrent Neural Networks

Contact Info

Product

Resources

About