3DNN: Viewpoint Invariant 3D Geometry Matching for Scene Understanding

Satkin, Scott; Hebert, Martial

doi:10.1109/iccv.2013.235

Cited by 25 publications

(23 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our model can be understood as a novel way to extend this framework to 3D. There are also works on using CAD models for training [31][32][33][34][35][36], but they are not for depth images.…”

Section: Related Work and Discussionmentioning

confidence: 99%

Sliding Shapes for 3D Object Detection in Depth Images

Song

Xiao

2014

Lecture Notes in Computer Science

333

284

View full text Add to dashboard Cite

Abstract. The depth information of RGB-D sensors has greatly simplified some common challenges in computer vision and enabled breakthroughs for several tasks. In this paper, we propose to use depth maps for object detection and design a 3D detector to overcome the major difficulties for recognition, namely the variations of texture, illumination, shape, viewpoint, clutter, occlusion, selfocclusion and sensor noises. We take a collection of 3D CAD models and render each CAD model from hundreds of viewpoints to obtain synthetic depth maps. For each depth rendering, we extract features from the 3D point cloud and train an Exemplar-SVM classifier. During testing and hard-negative mining, we slide a 3D detection window in 3D space. Experiment results show that our 3D detector significantly outperforms the state-of-the-art algorithms for both RGB and RGB-D images, and achieves about ×1.7 improvement on average precision compared to DPM and R-CNN. All source code and data are available online.

show abstract

Section: Related Work and Discussionmentioning

confidence: 99%

Sliding Shapes for 3D Object Detection in Depth Images

Song

Xiao

2014

Lecture Notes in Computer Science

333

284

View full text Add to dashboard Cite

show abstract

“…For every placement of a 3D object (which we call swapping) we calculate its geometric likelihood -that is the similarity between the image and the projected 3D model. This geometric likelihood is evaluated using the method of [22] where various geometrically meaningful image features, such as surface normal and clutter estimates, are combined in a learned linear model to output a single number as a geometric similarity score. In other words, the geometric similarity score is w T L, where each row of L is the output of a geometrically meaningful image feature 1 , and w is a learned weight vector.…”

Section: Approachmentioning

confidence: 99%

“…Secondly, we further the work of one of the most recent and state of the art works in the area of scene understanding, of Satkin et al [22], by creating a flexible yet robust framework for the incorporation of both semantic and appearance cues for the purpose of achieving better holistic 3D scene understanding.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Detailed 3D Model Driven Single View Scene Understanding

Rashid

Hebert

2014

2014 2nd International Conference on 3D Vision

Self Cite

View full text Add to dashboard Cite

We present a data driven approach to holistic scene understanding. From a single image of an indoor scene, our approach estimates its detailed 3D geometry, i.e. the location of its walls and floor, and the 3D appearance of its containing objects, as well as its semantic meaning, i.e. a prediction of what objects it contains. This is made possible by using large datasets of detailed 3D models alongside appearance based detectors. We first estimate the 3D layout of a room, and extrapolate 2D object detection hypotheses to three dimensions to form bounding cuboids. Cuboids are converted to detailed 3D models of the predicted semantic category. Combinations of 3D models are used to create a large list of layout hypotheses for each image -where each layout hypothesis is semantically meaningful and geometrically plausible. The likelihood of each layout hypothesis is ranked using a learned linear model -and the hypothesis with the highest predicted likelihood is the final predicted 3D layout. Our approach is able to recover the detailed geometry of scenes, provide precise segmentation of objects in the image plane, and estimate objects' pose in 3D.

show abstract

“…Despite the computational complexity involved in such tasks, new computer vision developments potentially can help facing such challenging tasks at manageable costs (Google, 2016). For example, computer vision systems could help improving the quality of the images taken in the presence of reflecting or occluding elements, such as windows and fences (Xue et al, 2015), compensating the viewpoint from which the image was captured in three-dimensional visual data processing (Satkin and Hebert, 2013), localizing the user in the surrounding environment, so that indoor and outdoor scenes can be represented in referenced spaces (Bettadapura et al, 2015), or yet capturing automatically the relationships among visual elements in real-world scenarios (Choi et al, 2015).…”

Section: Introductionmentioning

confidence: 99%

Bringing Vision-Based Measurements into our Daily Life: A Grand Challenge for Computer Vision Systems

Scharcanski

2016

Front. ICT

View full text Add to dashboard Cite

Bringing computer vision into our daily life has been challenging researchers in industry and in academia over the past decades. However, the continuous development of cameras and computing systems turned computer vision-based measurements into a viable option, allowing new solutions to known problems. In this context, computer vision is a generic tool that can be used to measure and monitor phenomena in wide range of fields. The idea of using vision-based measurements is appealing, since these measurements can be fast and easy to obtain and handle by humans. On the other hand, these vision-based measurements need to be at least as precise and accurate as those obtained by more traditional measuring systems. This article discusses the perspectives and challenges of introducing vision-based measurements into our daily life.

show abstract

3DNN: Viewpoint Invariant 3D Geometry Matching for Scene Understanding

Abstract: We present a new algorithm 3DNN (3D NearestNeighbor)

Cited by 25 publications

References 35 publications

Sliding Shapes for 3D Object Detection in Depth Images

Sliding Shapes for 3D Object Detection in Depth Images

Detailed 3D Model Driven Single View Scene Understanding

Bringing Vision-Based Measurements into our Daily Life: A Grand Challenge for Computer Vision Systems

Contact Info

Product

Resources

About