Image Region Labeling by Exploring Contextual Information of Visual Spatial and Semantic Concepts

Abstract

Abstract. Region Labeling is to automatically assign semantic labels to the corresponding image regions. Most of the previous works focus on exploiting low level visual features, particularly visual spatial contextual information, to address the problem. However, very few work explore high level semantic information of the whole image to deal with the problem. In this paper, we propose a new region labeling approach by integrating both visual spatial and semantic contextual information into a unified model. In our method, region labeling is regarded as a multi-class classification problem. For each semantic concept, we train a Conditional Random Field (CRF) model respectively. It consists of both the region grid sub-graph and the co-occurred semantic label sub-graph. In our model, the integration of the two kinds of contextual information brings reinforcement effect on the improvement of region labeling. The experiments are conducted on two commonly used benchmark datasets and the experimental results show that our method achieves the best performance compared with the strong baselines and the state-of-the-art methods.