A Multi-Task Network for Joint Specular Highlight Detection and Removal

Fu, Gang; Zhang, Qing; Zhu, Lei; Li, Ping; Xiao, Chunxia

doi:10.1109/cvpr46437.2021.00766

Cited by 69 publications

(84 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The extensive experiment results have strongly confirmed the effectiveness and superiority of our method. Our framework can be extended to handle more computer vision tasks such as highlight removal [9,11], which we take as the future work.…”

Section: Discussionmentioning

confidence: 99%

CANet: A Context-Aware Network for Shadow Removal

Chen¹,

Long²,

Zhang³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

In this paper, we propose a novel two-stage contextaware network named CANet for shadow removal, in which the contextual information from non-shadow regions is transferred to shadow regions at the embedded feature spaces. At Stage-I, we propose a contextual patch matching (CPM) module to generate a set of potential matching pairs of shadow and non-shadow patches. Combined with the potential contextual relationships between shadow and non-shadow regions, our well-designed contextual feature transfer (CFT) mechanism can transfer contextual information from non-shadow to shadow regions at different scales. With the reconstructed feature maps, we remove shadows at L and A/B channels separately. At Stage-II, we use an encoder-decoder to refine current results and generate the final shadow removal results. We evaluate our proposed CANet on two benchmark datasets and some real-world shadow images with complex scenes. Extensive experimental results strongly demonstrate the efficacy of our proposed CANet and exhibit superior performance to state-of-the-arts. Our source code is available at https://github.com/Zipei-Chen/CANet.

show abstract

Section: Discussionmentioning

confidence: 99%

CANet: A Context-Aware Network for Shadow Removal

Chen¹,

Long²,

Zhang³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…To reduce specular reflection in images, we adopted the network architecture from Fu et al [3] as the image enhancement stage to recover the image content from specular regions. This step improves the perception ability of the feature extraction module, enhancing the depth quality in texture-less specular regions.…”

Section: Specularity Removalmentioning

confidence: 99%

“…Previous representative solutions, such as the robust principal component analysis [23,12], have both robustness and computational efficiency issues. Although neural network solutions [2,3] have been discussed in the computer vision domain, little work for endoscopy surgery has collaborated with other perception applications. To our best knowledge, we firstly integrate a specularity removal network module in the pipeline to improve the quality and efficiency of depth estimation.…”

Section: Introductionmentioning

confidence: 99%

3D endoscopic depth estimation using 3D surface-aware constraints

Shang¹,

Wang²,

Liu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Robotic-assisted surgery allows surgeons to conduct precise surgical operations with stereo vision and flexible motor control. However, the lack of 3D spatial perception limits situational awareness during procedures and hinders mastering surgical skills in the narrow abdominal space. Depth estimation, as a representative perception task, is typically defined as an image reconstruction problem. In this work, we show that depth estimation can be reformed from a 3D surface perspective. We propose a loss function for depth estimation that integrates the surfaceaware constraints, leading to a faster and better convergence with the valid information from spatial information. In addition, camera parameters are incorporated into the training pipeline to increase the control and transparency of the depth estimation. We also integrate a specularity removal module to recover more buried image information. Quantitative experimental results on endoscopic datasets and user studies with medical professionals demonstrate the effectiveness of our method.

show abstract

“…Non-learning based approaches usually exploit appearance or statistical priors to separate specular reflection, including chromaticity-based models [Akashi and Okatani 2016;Shafer 1985;Tan and Ikeuchi 2005;Yang et al 2010], low-rank model [Guo et al 2018], and dark channel prior [Kim et al 2013]. Recently, data-driven methods [Fu et al 2021;Shi et al 2017;Wu et al 2020] train deep networks in a supervised manner. Shi et al [2017] train a CNN model using their proposed object-centric synthetic dataset.…”

Section: Related Workmentioning

confidence: 99%

“…Shi et al [2017] train a CNN model using their proposed object-centric synthetic dataset. Fu et al [2021] present a large real-world dataset for highlight removal, and introduce a multi-task network to detect and remove specular reflection. However, the reflection on floors is more complex than highlights, because it may reflect window textures and occupy a large region.…”

Section: Related Workmentioning

confidence: 99%