SUN RGB-D: A RGB-D scene understanding benchmark suite

Song, Shuran; Lichtenberg, Samuel P.; Xiao, Jianxiong

doi:10.1109/cvpr.2015.7298655

Cited by 1,580 publications

(1,234 citation statements)

References 49 publications

Supporting

Mentioning

1,163

Contrasting

Unclassified

Order By: Relevance

“…For more comprehensive comparisons, besides these 20 14 , from top left to bottom right is the chronological order from the video. The curves under the images are the accelerometer data at 50 Hz of devices attached to the knife, the mixing spoon, the small spoon, the peeler, the glass, the oil bottle, and the pepper dispenser mentioned datasets above, another 26 extra RGB-D datasets for different applications are also added into the tables: Birmingham University Objects, Category Modeling RGB-D [104], Cornell Activity [47,92], Cornell RGB-D [48], DGait [12], Daily Activities with occlusions [1], Heidelberg University Scenes [63], Microsoft 7-scenes [78], MobileRGBD [96], MPII Multi-Kinect [93], MSR Action3D Dataset [97], MSR 3D Online Action [103], MSRGesture3D [50], DAFT [31], Paper Kinect [70], RGBD-HuDaAct [68], Stanford Scene Object [44], Stanford 3D Scene [105], Sun3D [101], SUN RGB-D [82], TST Fall Detection [28], UTD-MHAD [14], Vienna University Technology Object [2], Willow Garage [99], Workout SU-10 exercise [67] and 3D-Mask [24]. In addition, we name those datasets without original names by means of creation place or applications.…”

Section: Discussionmentioning

confidence: 99%

RGB-D datasets using microsoft kinect or similar sensors: a survey

Cai

Han

Liu

et al. 2016

Multimed Tools Appl

130

View full text Add to dashboard Cite

RGB-D data has turned out to be a very useful representation of an indoor scene for solving fundamental computer vision problems. It takes the advantages of the color image that provides appearance information of an object and also the depth image that is immune to the variations in color, illumination, rotation angle and scale. With the invention of the low-cost Microsoft Kinect sensor, which was initially used for gaming and later became a popular device for computer vision, high quality RGB-D data can be acquired easily. In recent years, more and more RGB-D image/video datasets dedicated to various applications have become available, which are of great importance to benchmark the stateof-the-art. In this paper, we systematically survey popular RGB-D datasets for different applications including object recognition, scene classification, hand gesture recognition, 3D-simultaneous localization and mapping, and pose estimation. We provide the insights into the characteristics of each important dataset, and compare the popularity and the difficulty of those datasets. Overall, the main goal of this survey is to give a comprehensive description about the available RGB-D datasets and thus to guide researchers in the selection of suitable datasets for evaluating their algorithms.

show abstract

Section: Discussionmentioning

confidence: 99%

RGB-D datasets using microsoft kinect or similar sensors: a survey

Cai

Han

Liu

et al. 2016

Multimed Tools Appl

130

View full text Add to dashboard Cite

show abstract

“…Existing indoor datasets such as the NYU Depth V2 dataset [19] and the SUN RGBD dataset [20] are largely composed by images of cluttered rooms, which are of less interest for our purposes.…”

Section: Implementation and Training A Training Datasetmentioning

confidence: 99%

“…Examples images are shown in Figure 3. It contains 967 images from three sources: 349 images from the SUN RGBD [20] (category "corridor"); 327 images from SUN database [21] (category "corridor") and 291 images from self-collected video taken around the Carnegie Mellon University campus. For the SUN database images, we used annotations where available, and manually annotated an extra 250 images using LabelMe [22].…”

Section: Implementation and Training A Training Datasetmentioning

confidence: 99%

Real-time 3D scene layout from a single image using Convolutional Neural Networks

Yang

Maturana

Scherer

2016

2016 IEEE International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

We consider the problem of understanding the 3D layout of indoor corridor scenes from a single image in real time. Identifying obstacles such as walls is essential for robot navigation, but also challenging due to the diversity in structure, appearance and illumination of real-world corridor scenes. Many current single-image methods make Manhattanworld assumptions, and break down in environments that do not meet this mold. They also may require complicated handdesigned features for image segmentation or clear boundaries to form certain building models. In addition, most cannot run in real time.In this paper, we propose to combine machine learning with geometric modelling to build a simplified 3D model from a single image. We first employ a supervised Convolutional Neural Network (CNN) to provide a dense, but coarse, geometric class labelling of the scene. We then refine this labelling with a fully connected Conditional Random Field (CRF). Finally, we fit line segments along wall-ground boundaries and "pop up" a 3D model using geometric constraints.We assemble a dataset of 967 labelled corridor images. Our experiments on this dataset and another publicly available dataset show our method outperforms other single image scene understanding methods in pixelwise accuracy while labelling images at over 15 Hz.

show abstract

“…We use the SUN RGB-D dataset from Song et al [31]. This dataset contains 10335 RGB images with depth maps, as well as detailed annotations for more than 1000 objects in the form of 2D and 3D bounding boxes.…”

Section: Methodsmentioning

confidence: 99%

“…We circumvent this problem by leveraging the fast RCNN detector of [14] with object proposals generated by Selective Search [36]. In detail, we finetune the ImageNet model from [14] to SUN RGB-D, using the same 19 objects as in [31]. We then run the detector on all images from our training split and keep the proposals with detection scores > 0.7 and a sufficient overlap (measured by the IoU >0.5) with the 2D ground-truth bounding boxes.…”

Section: Monitormentioning

confidence: 99%

AGA: Attribute-Guided Augmentation

Dixit

Kwitt

Niethammer

et al. 2017

2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

101

View full text Add to dashboard Cite

We consider the problem of data augmentation, i.e., generating artificial samples to extend a given corpus of training data. Specifically, we propose attributed-guided augmentation (AGA) which learns a mapping that allows synthesis of data such that an attribute of a synthesized sample is at a desired value or strength. This is particularly interesting in situations where little data with no attribute annotation is available for learning, but we have access to an external corpus of heavily annotated samples. While prior works primarily augment in the space of images, we propose to perform augmentation in feature space instead. We implement our approach as a deep encoder-decoder architecture that learns the synthesis function in an end-to-end manner. We demonstrate the utility of our approach on the problems of (1) one-shot object recognition in a transferlearning setting where we have no prior knowledge of the new classes, as well as (2) object-based one-shot scene recognition. As external data, we leverage 3D depth and pose information from the SUN RGB-D dataset. Our experiments show that attribute-guided augmentation of highlevel CNN features considerably improves one-shot recognition performance on both problems.

show abstract

SUN RGB-D: A RGB-D scene understanding benchmark suite

Cited by 1,580 publications

References 49 publications

RGB-D datasets using microsoft kinect or similar sensors: a survey

RGB-D datasets using microsoft kinect or similar sensors: a survey

Real-time 3D scene layout from a single image using Convolutional Neural Networks

AGA: Attribute-Guided Augmentation

Contact Info

Product

Resources

About