SAMP: Shape and Motion Priors for 4D Vehicle Reconstruction

Engelmann, Francis; Stückler, Jörg; Leibe, Bastian

doi:10.1109/wacv.2017.51

Cited by 41 publications

(25 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Usually, the vehicle states are also estimated jointly through a filtering process by considering the vehicle kinematics or dynamics [27] [28]. The vehicle shape can be reconstructed using stereo cameras [29] or monocular cameras [25][30] [31] through a sequence of algorithms for depth estimation, model fitting, and shape optimization. Our paper is based on the existing works for several individual computer vision tasks and we integrated them to a unified framework that extracts the location, speed, and vehicle shape in the 3D space.…”

Section: Related Workmentioning

confidence: 99%

CAROM -- Vehicle Localization and Traffic Scene Reconstruction from Monocular Cameras on Road Infrastructures

Lü¹,

Jammula²,

Como³

et al. 2021

Preprint

View full text Add to dashboard Cite

Traffic monitoring cameras are powerful tools for traffic management and essential components of intelligent road infrastructure systems. In this paper, we present a vehicle localization and traffic scene reconstruction framework using these cameras, dubbed as CAROM, i.e., "CARs On the Map". CAROM processes traffic monitoring videos and converts them to anonymous data structures of vehicle type, 3D shape, position, and velocity for traffic scene reconstruction and replay. Through collaborating with a local department of transportation in the United States, we constructed a benchmarking dataset containing GPS data, roadside camera videos, and drone videos to validate the vehicle tracking results. On average, the localization error is approximately 0.8 m and 1.7 m within the range of 50 m and 120 m from the cameras, respectively.

show abstract

Section: Related Workmentioning

confidence: 99%

CAROM -- Vehicle Localization and Traffic Scene Reconstruction from Monocular Cameras on Road Infrastructures

Lü¹,

Jammula²,

Como³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…However, the loss of information caused by decoupling detection and tracking may lead to sub-optimal solutions. Benefit from stereo images, the method [19] focuses on reconstructing the object using 3D shape and motion priors, and the method [42] exploits a dynamic object bundle adjustment (BA) approach which fuses temporal sparse feature correspondences and the semantic 3D measurement model to continuously track the object, while the performance on 3D localization for occluded objects is limited. From another aspect, Luo et al [49] encode 3D point clouds into 3D voxel representations and jointly reason about 3D detection, tracking and motion forecasting so that it is more robust to occlusion as well as sparse data at range.…”

Section: D Object Trackingmentioning

confidence: 99%

Know Your Surroundings: Panoramic Multi-Object Tracking by Multimodality Collaboration

He¹,

Yu²,

Han³

et al. 2021

Preprint

View full text Add to dashboard Cite

In this paper, we focus on the multi-object tracking (MOT) problem of automatic driving and robot navigation. Most existing MOT methods track multiple objects using a singular RGB camera, which are prone to camera field-ofview and suffer tracking failures in complex scenarios due to background clutters and poor light conditions. To meet these challenges, we propose a MultiModality PAnoramic multi-object Tracking framework (MMPAT), which takes both 2D panorama images and 3D point clouds as input and then infers target trajectories using the multimodality data. The proposed method contains four major modules, a panorama image detection module, a multimodality data fusion module, a data association module and a trajectory inference model. We evaluate the proposed method on the JRDB dataset, where the MMPAT achieves the top performance in both the detection and tracking tasks and significantly outperforms state-of-the-art methods by a large margin (15.7 and 8.5 improvement in terms of AP and MOTA, respectively).

show abstract

“…In [8] the authors use stereo depth and run a detector to initialize instances which are further optimized for pose and shape via SDF priors. In a follow-up work [9] the authors extended the framework with temporal priors to simultaneously recover smooth shapes and pose trajectories. Slightly related, [36] explores 3D object completion via a shape space and LIDAR as weak supervision with a probabilistic formulation.…”

Section: Related Workmentioning

confidence: 99%

Autolabeling 3D Objects with Differentiable Rendering of SDF Shape Priors

Захаров¹,

Kehl²,

Bhargava³

et al. 2019

Preprint

View full text Add to dashboard Cite

We present an automatic annotation pipeline to recover 9D cuboids and 3D shape from pre-trained off-the-shelf 2D detectors and sparse LIDAR data. Our autolabeling method solves this challenging ill-posed inverse problem by relying on learned shape priors and optimization of geometric and physical parameters. To that end, we propose a novel differentiable shape renderer over signed distance fields (SDF), which we leverage in combination with normalized object coordinate spaces (NOCS). Initially trained on synthetic data to predict shape and coordinates, our method uses these predictions for projective and geometrical alignment over real samples. We also propose a curriculum learning strategy, iteratively retraining on samples of increasing difficulty for subsequent self-improving annotation rounds. Our experiments on the KITTI3D dataset show that we can recover a substantial amount of accurate cuboids, and that these autolabels can be used to train 3D vehicle detectors with state-of-the-art results. We will make the code publicly available soon.

show abstract

SAMP: Shape and Motion Priors for 4D Vehicle Reconstruction

Cited by 41 publications

References 16 publications

CAROM -- Vehicle Localization and Traffic Scene Reconstruction from Monocular Cameras on Road Infrastructures

CAROM -- Vehicle Localization and Traffic Scene Reconstruction from Monocular Cameras on Road Infrastructures

Know Your Surroundings: Panoramic Multi-Object Tracking by Multimodality Collaboration

Autolabeling 3D Objects with Differentiable Rendering of SDF Shape Priors

Contact Info

Product

Resources

About