Real-time 3D scene layout from a single image using Convolutional Neural Networks

Abstract

We consider the problem of understanding the 3D layout of indoor corridor scenes from a single image in real time. Identifying obstacles such as walls is essential for robot navigation, but also challenging due to the diversity in structure, appearance and illumination of real-world corridor scenes. Many current single-image methods make Manhattanworld assumptions, and break down in environments that do not meet this mold. They also may require complicated handdesigned features for image segmentation or clear boundaries to form certain building models. In addition, most cannot run in real time.In this paper, we propose to combine machine learning with geometric modelling to build a simplified 3D model from a single image. We first employ a supervised Convolutional Neural Network (CNN) to provide a dense, but coarse, geometric class labelling of the scene. We then refine this labelling with a fully connected Conditional Random Field (CRF). Finally, we fit line segments along wall-ground boundaries and "pop up" a 3D model using geometric constraints.We assemble a dataset of 967 labelled corridor images. Our experiments on this dataset and another publicly available dataset show our method outperforms other single image scene understanding methods in pixelwise accuracy while labelling images at over 15 Hz.

Obstacle Avoidance Strategy for Mobile Robot Based on Monocular Camera

“…The authors use different scales to refine their predictions significantly. According to Yang et al [28], this architecture starts with a low-resolution, rough output prediction and refines it by fusing with prior layers to give both local and global reasoning. VGG-16 is introduced as a convolutional and max pooling layer-based encoder block in this case, as shown in Figure 1.…”

“…Network parameters are optimized for binary cross-entropy via one of the SGD variants' parameter learning strategies. In contrast to the suggestion made by Yang et al [28], our network's capabilities were increased through data augmentation, in which the original versions of the training images are used. The training process is sped signifi cantly thanks to the extensive use of transfer learning.…”

“…Additionally, practically eliminating noise will considerably improve the quality of the prediction. The segmentation noise filtering method is finalized using the probabilistic models described in [28]. Network parameters are optimized for binary cross-entropy via one of the SGD variants' parameter learning strategies.…”

Feature-based visual simultaneous localization and mapping: a survey

“…To detect planes in images, model fitting algorithms, such as RANSAC, are employed. It is also possible to combine modeling and a convolutional neural network (CNN) to identify planes, such as walls, in an image [136]. As for high-level features, several techniques were proposed for detecting objects and semantically labeling them in images including, but not limited to, conditional random fields (CRFs) [51], support vector machines (SVMs) [30], and deep neural networks (for example: single shot multi-box detector [74] and you only look once (YOLO) [104]).…”

Sigma-FP: Robot Mapping of 3D Floor Plans With an RGB-D Camera Under Uncertainty

“…The generation of 3D floor plans is usually performed by extracting primitive shapes such as cuboids or planes from data acquired with on board cameras [6], [7] and/or range sensors [8], [9]. The limitations of current techniques involve (i) the requirement of a large amount of input data (e.g.…”