3-D reconstruction of a dynamic environment with a fully calibrated background for traffic scenes

Abstract

Abstract-Vision-based traffic surveillance systems are more and more employed for traffic monitoring, collection of statistical data and traffic control. We present an extension of such a system that additionally uses the captured image content for 3-D scene modeling and reconstruction. A basic goal of surveillance systems is to get a good coverage of the observed area with as few cameras as possible to keep the costs low. Therefore, the 3-D reconstruction has to be done from only a few original views with limited overlap and different lighting conditions. To cope with these specific restrictions we developed a model-based 3-D reconstruction scheme that exploits a priori knowledge about the scene. The system is fully calibrated offline by estimating camera parameters from measured 3-D-2-D correspondences. Then the scene is divided into static parts, which are modeled offline and dynamic parts, which are processed online. Therefore, we segment all views into moving objects and static background. The background is modeled as multitexture planes using the original camera textures. Moving objects are segmented and tracked in each view. All segmented views of a moving object are combined to a 3-D object, which is positioned and tracked in 3-D. Here we use predefined geometric primitives and map the original textures onto them. Finally the static and dynamic elements are combined to create the reconstructed 3-D scene, where the user can freely navigate, i.e., choose an arbitrary viewpoint and direction. Additionally, the system allows analyzing the 3-D properties of the scene and the moving objects.

Traffic Monitoring using Multiple Cameras, Homographies and Multi-Hypothesis Tracking

“…Preliminary results of the TRAVIS system have been obtained using as input a stereo (two-view) traffic monitoring sequence provided by HHI [6]. Original images are shown in Figures 2(a-b) and the estimated foreground probability maps are presented in Figures 2(c-d) respectively.…”

Development of MPEG standards for 3D and free viewpoint video

“…This can be extended to Layered Depth Images [38] where multiple color and depth values are stored in consecutively ordered depth layers. Closer to the geometry-based end of the spectrum we can find methods that use view-dependent geometry and/or view dependent texture [23], [31]. Surface light-fields combine the idea of light-fields with an explicit 3D model [39], [40].…”

“…The first multi-view signal processing step consists in segmentation of the objects of interest, i.e. those that shall be reconstructed in 3D [23]. Although a huge effort has been put into this, segmentation is still and error prone task.…”

“…Since 3D objects may appear very different from different directions, depending on light sources and reflectance properties static texturing may lead to poor rendering results. This can be overcome by view-dependent texture mapping since multiple views of the object are available [23], [31]. The available textures from the cameras are weighted depending on the distance to the virtual viewpoint and blended over the object.…”

Tracking and reconstruction of vehicles for accurate position estimation

“…In [9] a system for tracking pedestrians and cars was presented which is based on object detectors. A system for making 3D shape reconstruction for traffic surveillance with multiple cameras is presented in [10]. To do this predefined 3D models of cars are used.…”