Facilitating and Exploring Planar Homogeneous Texture for Indoor Scene Understanding

Abstract

Indoor scenes tend to be abundant with planar homogeneous texture, manifesting as regularly repeating scene elements along a plane. In this work, we propose to exploit such structure to facilitate high-level scene understanding. By robustly fitting a texture projection model to optimal dominant frequency estimates in image patches, we arrive at a projective-invariant method to localize such semantically meaningful regions in multi-planar scenes. The recovered projective parameters also allow an affine-ambiguous rectification in real-world images marred with outliers, room clutter, and photometric severities. Qualitative and quantitative results show our method outperforms existing representative work for both rectification and detection. We then explore the potential of homogeneous texture for two indoor scene understanding tasks. In scenes where vanishing points cannot be reliably detected, or the Manhattan assumption is not satisfied, homogeneous texture detected by the proposed approach provides alternative cues to obtain an indoor scene geometric layout. Second, low-level feature descriptors extracted upon affine rectification of detected texture are found to be not only class-discriminative but also complementary to features without rectification, improving recognition performance on the MIT Indoor67 benchmark.

3G structure for image caption generation

“…Moreover, attention mechanism is widely used in other computer vision and natural language processing tasks [32,33,34,35]. RNN (1)   RNN (2) RNN (1)…”

“…RNN (1) 1 t y − In this paper, we retain the attention part as the local image feature. To improve the performance of the language part, gated feedback connecting strategy is used for stacking the LSTM.…”

“…Image caption generation aims to automatically generate a natural language sentence to describe the content of a given image. It is a vital task of scene understanding which is one of the fundamental goals of computer vision and artificial intelligence [1,2]. However, image caption generation is a challenging task.…”