Towards Robust Human Trajectory Prediction in Raw Videos

Abstract

Human trajectory prediction has received increased attention lately due to its importance in applications such as autonomous vehicles and indoor robots. However, most existing methods make predictions based on human-labeled trajectories and ignore the errors and noises in detection and tracking. In this paper, we study the problem of human trajectory forecasting in raw videos, and show that the prediction accuracy can be severely affected by various types of tracking errors. Accordingly, we propose a simple yet effective strategy to correct the tracking failures by enforcing prediction consistency over time. The proposed "re-tracking" algorithm can be applied to any existing tracking and prediction pipelines. Experiments on public benchmark datasets demonstrate that the proposed method can improve both tracking and prediction performance in challenging real-world scenarios. The code and data are available at https://git.io/retracking-prediction.

MTP: Multi-Hypothesis Tracking and Prediction for Reduced Error Propagation

“…Here, k = 1 as Social-LSTM is a deterministic prediction approach. Both the MTP and re-tracking approaches [37] show improvement over STP when using past tracklets as inputs, with MTP showing a slightly larger improvement. Importantly, the re-tracking approach, which is single-hypothesis-based, and the MTP framework are complementary, and indeed combining the two approaches (in the last row in Table V) further improves prediction performance!…”

“…Comparison with Concurrent Work [37]. As discussed in Section II, an unpublished work [37] has proposed a single-hypothesis-based re-tracking solution to mitigate the impact of tracking errors on prediction.…”

“…As discussed in Section II, an unpublished work [37] has proposed a single-hypothesis-based re-tracking solution to mitigate the impact of tracking errors on prediction. Though [37] has open-sourced its code, direct comparison between MTP and [37] is not immediate as [37] is implemented on a 2D (rather than 3D, as in this paper) detection, tracking, and prediction pipeline, that is, using MaskRCNN [45] for detection in images, using SORT [43] for 2D MOT, and using Social-LSTM [44] for bird's eye view trajectory prediction. The method is evaluated on a multi-camera dataset, WILDTRACK [46].…”

“…The method is evaluated on a multi-camera dataset, WILDTRACK [46]. To ensure a fair comparison, we add our MHDA and trajectory sampling modules to the SORT+Social-LSTM pipeline implemented by [37] and carry out the evaluation on WILD-TRACK. Note that, as [37] did not release detection results by the time this paper was submitted, we use the Detectron2 implementation of MaskRCNN with an X101-FPN backbone [47] to generate detections.…”

“…Finally, a concurrent and unpublished work [37] has also recognized the importance of understanding how tracking errors can impact prediction. Our paper provides a comprehensive quantitative analysis that corroborates the qualitative findings in [37], and we propose to leverage MHDA to more robustly accounting for tracking errors, while the solution method in [37] is still single-hypothesis-based.…”