Object detection and classification from large‐scale cluttered indoor scans

Mattausch, Oliver; Panozzo, Daniele; Mura, Claudio; Sorkine‐Hornung, Olga; Pajarola, Renato

doi:10.1111/cgf.12286

Cited by 96 publications

(87 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Thus, object-level structure analysis is investigated to match the captured objects to the high-quality 3D models in databases for improved reconstruction quality [Nan et al 2012;Shao et al 2012;Salas-Moreno et al 2013], while object repetition is explored to speed up large scale indoor scene reconstruction [Kim et al 2012;Mattausch et al 2014]. In contrast, our approach does not rely on a 3D model database, but performs online repeated object analysis to simultaneously reduce the scanning burden by providing real-time guidance to the user and improve the reconstruction quality through local volume fusion.…”

Section: Related Workmentioning

confidence: 99%

Online Structure Analysis for Real-Time Indoor Scene Reconstruction

Zhang

Tong

et al. 2015

ACM Trans. Graph.

View full text Add to dashboard Cite

Figure 1: A lab scene (100m 2 ) reconstructed on-the-fly. The complete scan finished within 70 minutes. Top left: the final reconstructed model by our system, which contains planar regions (polygons) and extracted meshes for separate objects, created and refined progressively in real-time by our online analysis procedure during the scan. Top right: the result of the plane/object labeling procedure, which provides the segmentation of planes and objects for the online analysis. The colors distinguish planar regions and objects by their labels in the volumetric data structure. Bottom: close-up views of the reconstructed scene. AbstractWe propose a real-time approach for indoor scene reconstruction. It is capable of producing a ready-to-use 3D geometric model even while the user is still scanning the environment with a consumer depth camera. Our approach features explicit representations of planar regions and non-planar objects extracted from the noisy feed of the depth camera, via an online structure analysis on the dynamic, incomplete data. The structural information is incorporated into the volumetric representation of the scene, resulting in a seamless integration with KinectFusion's global data structure and an efficient implementation of the whole reconstruction process. Moreover, heuristics based on rectilinear shapes in typical indoor scenes effectively eliminate camera tracking drift and further improve reconstruction accuracy. The instantaneous feedback enabled by our on-the-fly structure analysis, including repeated object recognition, allows the user to selectively scan the scene and produce high fidelity large-scale models efficiently. We demonstrate the capability of our system with real-life examples. *

show abstract

Section: Related Workmentioning

confidence: 99%

Online Structure Analysis for Real-Time Indoor Scene Reconstruction

Zhang

Tong

et al. 2015

ACM Trans. Graph.

View full text Add to dashboard Cite

show abstract

“…Hence, in recent years, significant efforts have gone towards capturing and parsing 3D scenes. Approaches include using classifiers on scene objects [Schlecht and Barnard 2009;Xiong and Huber 2010;Anand et al 2013;Koppula et al 2011;Silberman et al 2012], interactive 3D modeling from raw RGBD scans [Shao et al 2012], interleaving segmentation and classification [Nan et al 2012], unsupervised algorithms to identify and consolidate scans [Kim et al 2012;Mattausch et al 2014], dynamic reconstruction [Mitra et al 2007], proxy geometry based scene understanding [Lafarge and Alliez 2013;Monszpart et al 2015], or studying the spatial layout of scenes Lee et al 2010;Hartley et al 2012].…”

Section: Related Workmentioning

confidence: 99%

Smash

2016

View full text Add to dashboard Cite

Figure 1: Starting from an input video of a collision sequence behind a curtain (top), SMASH reconstructs an accurate physically valid collision (bottom) using laws of rigid body physics for regularization. Note the reconstructed spin (i.e., angular velocity) of the objects. AbstractCollision sequences are commonly used in games and entertainment to add drama and excitement. Authoring even two body collisions in the real world can be difficult, as one has to get timing and the object trajectories to be correctly synchronized. After tedious trial-and-error iterations, when objects can actually be made to collide, then they are difficult to capture in 3D. In contrast, synthetically generating plausible collisions is difficult as it requires adjusting different collision parameters (e.g., object mass ratio, coefficient of restitution, etc.) and appropriate initial parameters. We present SMASH to directly read off appropriate collision parameters directly from raw input video recordings. Technically we enable this by utilizing laws of rigid body collision to regularize the problem of lifting 2D trajectories to a physically valid 3D reconstruction of the collision. The reconstructed sequences can then be modified and combined to easily author novel and plausible collisions. We evaluate our system on a range of synthetic scenes and demonstrate the effectiveness of our method by accurately reconstructing several complex real world collision events.

show abstract

“…Although certain popular public localization datasets (Dubrovnik [13], Rome [13], Vienna [11] and Aachen [18]) use 2D images as a query, these do not fit our need for capturing a short video as a query. Mattausch et al [17] scanned rooms using microCT to build 3D room models. As there are no features associated with each point, this dataset is not suitable for testing our method.…”

Section: Datasetmentioning

confidence: 99%

Localize Me Anywhere, Anytime: A Multi-task Point-Retrieval Approach

Yan

Ren

et al. 2015

2015 IEEE International Conference on Computer Vision (ICCV)

View full text Add to dashboard Cite

Image-based localization is an essential complement to GPS localization. Current image-based localization methods are based on either 2D-to-3D or 3D-to-2D to find the correspondences, which ignore the real scene geometric attributes. The main contribution of our paper is that we use a 3D model reconstructed by a short video as the query to realize 3D-to-3D localization under a multi-task point retrieval framework. Firstly, the use of a 3D model as the query enables us to efficiently select location candidates. Furthermore, the reconstruction of 3D model exploits the correlation among different images, based on the fact that images captured from different views for SfM share information through matching features. By exploring shared information (matching features) across multiple related tasks (images of the same scene captured from different views), the visual feature's view-invariance property can be improved in order to get to a higher point retrieval accuracy. More specifically, we use multi-task point retrieval framework to explore the relationship between descriptors and the 3D points, which extracts the discriminant points for more accurate 3D-to-3D correspondences retrieval. We further apply multi-task learning (MTL) retrieval approach on thermal images to prove that our MTL retrieval framework also provides superior performance for the thermal domain. This application is exceptionally helpful to cope with the localization problem in an environment with limited or even no light sources.

show abstract

Object detection and classification from large‐scale cluttered indoor scans

Cited by 96 publications

References 35 publications

Online Structure Analysis for Real-Time Indoor Scene Reconstruction

Online Structure Analysis for Real-Time Indoor Scene Reconstruction

Smash

Localize Me Anywhere, Anytime: A Multi-task Point-Retrieval Approach

Contact Info

Product

Resources

About