Semantic Guided Single Image Reflection Removal

Abstract

Reflection is common when we see through a glass window, which not only is a visual disturbance but also influences the performance of computer vision algorithms. Removing the reflection from a single image, however, is highly ill-posed since the color at each pixel needs to be separated into two values belonging to the clear background and the reflection respectively. To solve this, existing methods use additional priors such as reflection layer smoothness, double reflection effect, and color consistency to distinguish the two layers. However, these low-level priors may not be consistently valid in real cases. In this paper, inspired by the fact that human beings can separate the two layers easily by recognizing the objects and understanding the scene, we propose to use the object semantic cue, which is high-level information, as the guidance to help reflection removal. Based on the data analysis, we develop a multi-task end-to-end deep learning method with a semantic guidance component, to solve reflection removal and semantic segmentation jointly. Extensive experiments on different datasets show significant performance gain when using high-level object-oriented information. We also demonstrate the application of our method to other computer vision tasks.