Stereoscopic Scene Flow Computation for 3D Motion Understanding

Wedel, Andreas; Brox, Thomas; Vaudrey, Tobi; Rabe, Clemens; Franke, Uwe; Cremers, Daniel

doi:10.1007/s11263-010-0404-0

Cited by 161 publications

(149 citation statements)

References 39 publications

Supporting

Mentioning

148

Contrasting

Unclassified

Order By: Relevance

“…Dense 3D scene flow aims at the concurrent 3D reconstruction and motion estimation in dynamic scenes [Huguet andDevernay, 2007, Wedel andCremers, 2011]. The dense depth available with RGB-D sensors can simplify scene flow estimation.…”

Section: Related Workmentioning

confidence: 99%

Efficient Dense Rigid-Body Motion Segmentation and Estimation in RGB-D Video

Stückler

Behnke

2015

Int J Comput Vis

View full text Add to dashboard Cite

Motion is a fundamental grouping cue in video. Many current approaches to motion segmentation in monocular or stereo image sequences rely on sparse interest points or are dense but computationally demanding. We propose an efficient expectationmaximization (EM) framework for dense 3D segmentation of moving rigid parts in RGB-D video. Our approach segments images into pixel regions that undergo coherent 3D rigid-body motion. Our formulation treats background and foreground objects equally and poses no further assumptions on the motion of the camera or the objects than rigidness. While our EM-formulation is not restricted to a specific image representation, we supplement it with efficient image representation and registration for rapid segmentation of RGB-D video. In experiments, we demonstrate that our approach recovers segmentation and 3D motion at good precision.

show abstract

Section: Related Workmentioning

confidence: 99%

Efficient Dense Rigid-Body Motion Segmentation and Estimation in RGB-D Video

Stückler

Behnke

2015

Int J Comput Vis

View full text Add to dashboard Cite

show abstract

“…Three-dimensional (3D) tracking of objects became feasible using scene flow in images. Scene flow combines stereo matching and optical flow to estimate the 3D motion of the pixels in a scene [5][6][7]. However, scene flow is still not reliable for critical applications, and researchers continue working on improving the results of scene flow algorithms.…”

Section: Literature Reviewmentioning

confidence: 99%

Tracking 3D Moving Objects Based on GPS/IMU Navigation Solution, Laser Scanner Point Cloud and GIS Data

Hosseinyalamdary

Balazadegan

Toth

2015

IJGI

View full text Add to dashboard Cite

Abstract:Monitoring vehicular road traffic is a key component of any autonomous driving platform. Detecting moving objects, and tracking them, is crucial to navigating around objects and predicting their locations and trajectories. Laser sensors provide an excellent observation of the area around vehicles, but the point cloud of objects may be noisy, occluded, and prone to different errors. Consequently, object tracking is an open problem, especially for low-quality point clouds. This paper describes a pipeline to integrate various sensor data and prior information, such as a Geospatial Information System (GIS) map, to segment and track moving objects in a scene. We show that even a low-quality GIS map, such as OpenStreetMap (OSM), can improve the tracking accuracy, as well as decrease processing time. A bank of Kalman filters is used to track moving objects in a scene. In addition, we apply non-holonomic constraint to provide a better orientation estimation of moving objects. The results show that moving objects can be correctly detected, and accurately tracked, over time, based on modest quality Light Detection And Ranging (LiDAR) data, a coarse GIS map, and a fairly accurate Global Positioning System (GPS) and Inertial Measurement Unit (IMU) navigation solution.

show abstract

“…Conversely, [18,13] present a framework for computation of scene flow that separates stereo matching from scene flow computation. They argue that it is an advantage since the user is free to choose stereo and optical flow algorithm with best properties.…”

Section: Stereo and Scene Flowmentioning

confidence: 99%

“…In a similar fashion [3] describe a stereo algorithm with joint estimation of scene flow based on seed growing. The scene flow is grown from stereo matches from previous frame, then the scene flow is used to predict matching in the next frame.Conversely, [18,13] present a framework for computation of scene flow that separates stereo matching from scene flow computation. They argue that it is an advantage since the user is free to choose stereo and optical flow algorithm with best properties.…”

mentioning

confidence: 99%

Real-time global prediction for temporally stable stereo

Dobias

Šára

2011

2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops)

View full text Add to dashboard Cite

We present a method for calculation of disparity maps from stereo sequences. Disparity map from previous frame is first transferred to the new frame using estimated motion of the calibrated stereo rig. The predicted disparities are validated for the new frame and areas where prediction failed are matched with a traditional stereo matching algorithm. This method produces very fast and temporally stable stereo matching suitable for real-time applications even on non-parallel hardware. IntroLast decade marked an increasing interest of researchers in stereo matching of image sequences. This task comes up mainly in automotive industry, 3D TV technologies and robotics.Using classical stereo algorithms (designed for standalone image pairs) on stereo sequences is not sufficient for these cases because of several issues. First of all, the resulting disparity maps are not temporally consistent -most methods exhibit unwanted flicker between the frames of the sequence. Second, it is desired to lower the computational complexity to achieve higher processing framerates. Typically this means that algorithms have higher error rate because they search a smaller volume in disparity space or do other simplifications. Finally, additional temporal information such as ego-motion or 3D scene flow may be extracted. Related Work Spacetime StereoEarly work on spacetime stereo [19,5] proposed extensions of spatial windows used for computation of matching costs to spatiotemporal windows, however they do not perform well with dynamic scenes. Their main advantage is that existing algorithms can be easily adapted to handle temporal dimension. Recent work from Richardt et al. [14] use spatiotemporal windows for the temporal variant of their algorithm (with addition of per-frame weights).Temporally stable stereo proposed in [15] considers image sequences as space-time volumes and the matching cost is based on similarity of spatiotemporal elements called stequels -optical flow is not explicitly computed to recover motion.A different approach is taken in [2] where temporal smoothing is applied as a post-processing of disparity maps using a median filter for each pixel over few adjacent frames. In order to cope with motion they compute optical flow between frames and trace the pixels over time.Min et al.[12] achieve temporal stability by adding a coherence function to the stereo matching cost to lower the matching cost in areas with small changes between frames. Stereo and Scene FlowThe concept of scene flow has been introduced in [17] as an extension of optical flow to temporal dimension. Some algorithms are designed to take advantage of joint calculation of disparity maps and disparity (scene) flow. For example,in [9] disparities are computed either using WTA or DP strategy, then the disparity flow is calculated using previous frame. Disparity prediction is done for the next frame and matching costs are updated to ensure temporal smoothness. In a similar fashion [3] describe a stereo algorithm with joint estimation of scene flow based o...

show abstract

Stereoscopic Scene Flow Computation for 3D Motion Understanding

Cited by 161 publications

References 39 publications

Efficient Dense Rigid-Body Motion Segmentation and Estimation in RGB-D Video

Efficient Dense Rigid-Body Motion Segmentation and Estimation in RGB-D Video

Tracking 3D Moving Objects Based on GPS/IMU Navigation Solution, Laser Scanner Point Cloud and GIS Data

Real-time global prediction for temporally stable stereo

Contact Info

Product

Resources

About