Automatic Annotation for Semantic Segmentation in Indoor Scenes

Abstract

Future advancements in robot autonomy and sophistication of robotics tasks rest on robust, efficient, and taskdependent semantic understanding of the environment. Semantic segmentation is the problem of simultaneous segmentation and categorization of a partition of sensory data. The majority of current approaches tackle this using multi-class segmentation and labeling in a Conditional Random Field (CRF) framework [19] or by generating multiple object hypotheses and combining them sequentially [2]. In practical settings, the subset of semantic labels that are needed depend on the task and particular scene and labelling every single pixel is not always necessary. We pursue these observations in developing a more modular and flexible approach to multi-class parsing of RGBD data based on learning strategies for combining independent binary object-vsbackground segmentations in place of the usual monolithic multilabel CRF approach. Parameters for the independent binary segmentation models can be learned very efficiently, and the combination strategy-learned using reinforcement learningcan be set independently and can vary over different tasks and environments. Accuracy is comparable to state-of-art methods on a subset of the NYU-V2 dataset of indoor scenes [24] , while providing additional flexibility and modularity.

Reinforcement learning in autonomous driving

“…Md. Alimoor Reza and Jana Kosecka proposed a semantic segmentation model for indoor environment using RL, which is a more modular and flexible than traditional monolithic multi-label CRF approach, cutting down the workload for data processing and maximize performance on the benchmark dataset [6].…”

CODA: A Real-World Road Corner Case Dataset for Object Detection in Autonomous Driving

“…We would continue to enlarge CODA by exploring: 1) Use COPG on more real-world road scenes. 2) Since CODA is collected in the real world with high-quality annotation, we can generate more synthesized images following [1,16], or mine large-scale unlabeled road scene images in a semi-supervised manner [30,34,36,25]. Further discussion about potential negative societal impact of CODA are provided in Appendix C.…”

Learning deep representations for semantic image parsing: a comprehensive overview

“…Similarly to 2D LSTM [31], the long short-term memorized context fusion (LSTM-CF) model [29] was proposed to fuse 2D contextual information from photometric RGB and depth data. LSTM-CF can handle the challenges of severe occlusions and diverse appearances [63,44,64,43,65] for RGB-D indoor scene labeling. The photometric context is captured by stacking several convolutional layers, while the depth context is achieved by devising one LSTM layer that encodes both short-range and long-range spatial dependencies along the vertical direction.…”