Monocular model-based 3D vehicle tracking for autonomous vehicles in unstructured environment

Manz, Markus G.; Luettel, Thorsten; Hundelshausen, Felix von; Wuensche, Hans-Joachim

doi:10.1109/icra.2011.5979581

Cited by 37 publications

(17 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Therefore, we cannot use the evaluation methods of [12] and [14], in which a small number of cars are equipped with a measuring apparatus. Instead, we propose two methods to automatically evaluate our tracking velocity estimates on a large number of tracked objects.…”

Section: Resultsmentioning

confidence: 99%

Combining 3D Shape, Color, and Motion for Robust Anytime Tracking

Held

Levinson

Thrun

et al. 2014

Robotics: Science and Systems X

View full text Add to dashboard Cite

Abstract-Although object tracking has been studied for decades, real-time tracking algorithms often suffer from low accuracy and poor robustness when confronted with difficult, realworld data. We present a tracker that combines 3D shape, color (when available), and motion cues to accurately track moving objects in real-time. Our tracker allocates computational effort based on the shape of the posterior distribution. Starting with a coarse approximation to the posterior, the tracker successively refines this distribution, increasing in tracking accuracy over time. The tracker can thus be run for any amount of time, after which the current approximation to the posterior is returned. Even at a minimum runtime of 0.7 milliseconds, our method outperforms all of the baseline methods of similar speed by at least 10%. If our tracker is allowed to run for longer, the accuracy continues to improve, and it continues to outperform all baseline methods. Our tracker is thus anytime, allowing the speed or accuracy to be optimized based on the needs of the application. I. INTRODUCTIONMany robotics applications are limited in what they can achieve due to unreliable tracking estimates. For example, an autonomous vehicle driving past a row of parked cars should know if one of these cars is about to pull out into the lane. Current state-of-the-art trackers give noisy estimates of the velocity of these vehicles, which are difficult to track due to heavy occlusion and viewpoint changes. Additionally, without robust estimates of the velocity of nearby vehicles, merging onto or off of highways or changing lanes become formidable tasks. Similar issues will be encountered by any robot that must act autonomously in crowded, dynamic environments.Our tracker makes use of the full 3D shape of the object being tracked, which allows us to robustly track objects despite occlusions or changes in viewpoint. We place the 3D shape information in a probabilistic framework, in which we combine cues from shape, color, and motion. As we will show, adding color and motion information gives a large benefit to our system compared to using the 3D shape alone. This information is especially useful for distant objects or objects under heavy occlusions, when detailed 3D shape information may not be available.We make use of a grid-based method to sample velocities from the state space. Traditional grid-based approaches are too slow to track multiple objects in real-time. We are able to finely sample from a large grid in real-time through the use of a novel method called annealed dynamic histograms. We start by sampling from the state space at a coarse resolution, using an approximation to the posterior distribution over velocities. As the sampling resolution increases, we anneal this distribution,

show abstract

Section: Resultsmentioning

confidence: 99%

Combining 3D Shape, Color, and Motion for Robust Anytime Tracking

Held

Levinson

Thrun

et al. 2014

Robotics: Science and Systems X

View full text Add to dashboard Cite

show abstract

“…Many papers that combine tracking with segmentation and classification only report the binary accuracy of the segmentations and/or classifications, with no quantitative evaluation of the accuracy of the tracks' distance or velocity estimates [4], [5], [8]. Of those that do quantify distance or velocity accuracy, most do so on only one or a few tracks, either by equipping a single target vehicle with a measuring apparatus [14], or by hand-labeling objects or points from a small number of tracks [7], [9].…”

Section: Related Workmentioning

confidence: 98%

“…Other groups have upsampled range data to assign each pixel in an image to a 3D position [11]- [13]. Manz et al inferred the 3D location of a vehicle using only a monocular camera but with the knowledge of a previously-acquired 3D model [14]. Variants of ICP incorporating color have been explored, such as work by Men et al [15].…”

Section: Related Workmentioning

confidence: 99%

Precision tracking with sparse 3D and dense color 2D data

Held

Levinson

Thrun

2013

2013 IEEE International Conference on Robotics and Automation

View full text Add to dashboard Cite

Precision tracking is important for predicting the behavior of other cars in autonomous driving. We present a novel method to combine laser and camera data to achieve accurate velocity estimates of moving vehicles. We combine sparse laser points with a high-resolution camera image to obtain a dense colored point cloud. We use a color-augmented search algorithm to align the dense color point clouds from successive time frames for a moving vehicle, thereby obtaining a precise estimate of the tracked vehicle's velocity. Using this alignment method, we obtain velocity estimates at a much higher accuracy than previous methods. Through pre-filtering, we are able to achieve near real time results. We also present an online method for real-time use with accuracies close to that of the full method. We present a novel approach to quantitatively evaluate our velocity estimates by tracking a parked car in a local reference frame in which it appears to be moving relative to the ego vehicle. We use this evaluation method to automatically quantitatively evaluate our tracking performance on 466 separate tracked vehicles. Our method obtains a mean absolute velocity error of 0.27 m/s and an RMS error of 0.47 m/s on this test set. We can also qualitatively evaluate our method by building color 3D car models from moving vehicles. We have thus demonstrated that our method can be used for precision car tracking with applications to autonomous driving and behavior modeling.

show abstract

“…Notably, only few published approaches with manually generated 3D vehicle models had robust tracking results [4], [11]. …”

Section: A Related Workmentioning

confidence: 99%

Combining model- and template-based vehicle tracking for autonomous convoy driving

Fries

Luettel

Wuensche

2013

2013 IEEE Intelligent Vehicles Symposium (IV)

Self Cite

View full text Add to dashboard Cite

This paper presents a robust method for vehicle tracking with a monocular camera. A previously published model-based tracking method uses a particle filter which needs an initial vehicle hypothesis both at system start and in case of a tracking loss. We present a template-based solution using different features to estimate a 3D vehicle pose roughly but fast. Combining model-and template-based object tracking keeps the advantages of each algorithm: Precise estimation of the 3D vehicle pose and velocity combined with a fast (re-) initialization approach. The improved tracking system was evaluated while driving autonomously in urban and unstructured environments. The results show that poorly visible vehicles can be tracked during different weather conditions in real-time.

show abstract

Monocular model-based 3D vehicle tracking for autonomous vehicles in unstructured environment

Cited by 37 publications

References 26 publications

Combining 3D Shape, Color, and Motion for Robust Anytime Tracking

Combining 3D Shape, Color, and Motion for Robust Anytime Tracking

Precision tracking with sparse 3D and dense color 2D data

Combining model- and template-based vehicle tracking for autonomous convoy driving

Contact Info

Product

Resources

About