A Disocclusion Inpainting Framework for Depth-Based View Synthesis

Luo, Guibo; Zhu, Yuesheng; Weng, Zhenyu; Li, Zhaotian

doi:10.1109/tpami.2019.2899837

Cited by 30 publications

(14 citation statements)

References 61 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the future, we will investigate to improve the rendering efficiency while maintaining the visual quality. Moreover, the temporal correlation of adjacent frames will also be considered [47]. As the foreground object moves, the removed foreground region also changes, so the background content visible in different frames can be used to build a background model in the time domain.…”

Section: Discussionmentioning

confidence: 99%

Local Foreground Removal Disocclusion Filling Method for View Synthesis

Liang

Chen

et al. 2020

IEEE Access

View full text Add to dashboard Cite

View synthesis is an effective method to generate the contents of multiple views based on a limited number of reference views, which can be used in 2D to 3D conversion, free viewpoint video and multiview video rendering. Depth-image-based rendering (DIBR) is a practical technique to generate virtual view by using a 2D reference view and its depth image. However, a critical problem in DIBR process is that disocclusions might be produced in the synthesized image because the background occluded by the foreground objects in the reference view may be exposed in the virtual view. In this paper, a local foreground removal method is proposed for disocclusion filling. Morphology-based depth image preprocessing is performed before DIBR, aiming to correct the depth value of the ghosts and remove ghost artifacts. In the synthesized virtual image, pixels on the disocclusion edge are identified and classified. Then they are positioned in the reference image by inverse 3D warping. Local foreground regions that occlude the corresponding background are removed from both the reference image and its depth image based on the disocclusion edge pixels. Removed region is filled with surrounding background contents, and depth information is used in this process to prevent foreground penetration. The predicted background contents are warped to the disocclusion region, thereby achieving the hole filling. Experimental results show that the proposed method performs better than the other methods in disocclusion filling, and improves the subjective and objective quality of the synthesized view. In the evaluation results of PSNR, SSIM, FSIMc and VSI, our method improves by 0.

show abstract

Section: Discussionmentioning

confidence: 99%

Local Foreground Removal Disocclusion Filling Method for View Synthesis

Liang

Chen

et al. 2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…The dust and defects, which are strictly within B, change the gray-scale value of the corresponding pixels and cause information loss in the image of F 1 . Therefore, the hole filling algorithm [23] begins with a point…”

Section: The Total Number Of the Elements Inmentioning

confidence: 99%

Automatic Detection of Dispersed Defects in Resin Eyeglass Based on Machine Vision Technology

2020

View full text Add to dashboard Cite

This paper is concerned with detection of dispersed defects in the resin eyeglass. At present, manual detection is always used in industry, which inevitably causes impairing eyes and a high rate of false-negative detection. Statistics show that its average accuracy and the average detection time are about 85% and 10s, respectively. We for the first time propose an automatic approach to detection of dispersed defects in resin eyeglass, based on the machine vision technology. It is observed that the refractivity of the normal and the defective regions of an eyeglass are different, and thus the reflection image is also collected in our system in addition to the normal transmission image. Such an optical imaging system is modelled and its analysis shows the gray-scale gradient difference between the normal and defective regions in the acquired image is dramatically enhanced under the illumination of a point light source with our designed system. An image processing algorithm is then developed to reveal the above difference and detect dispersed defects in resin eyeglasses. Our simulation study verifies the proposed approach. Further, its experimental evaluation was carried out and the result was consistent with the simulation one, showing that its detection accuracy and the average detection time were 97.50% and 0.636s, respectively, which meet the requirements for online detection of dispersed defects. INDEX TERMS Automatic detection, dispersed defects, imaging analysis, machine vision, resin eyeglass.

show abstract

“…Super-resolution is also an inevitable component because online multimedia hosting sites often prefer to transmit low resolution images and videos with super-resolution performed on device so that the end user enjoys high resolution multimedia experience even on low bandwidth channel. Similarly, inpainting plays a crucial role in many downstream applications such as image editing, Augmented Reality [59], 'dis-occlusion' inpainting [42], [45] for novel view synthesis in a multi camera video capture setting [8] to be integrated with mobile Head Mounted Displays (HMD).…”

Section: Introductionmentioning

confidence: 99%

Lightweight Modules for Efficient Deep Learning Based Image Restoration

Lahiri

Bairagya

Bera

et al. 2021

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

Low level image restoration is an integral component of modern artificial intelligence (AI) driven camera pipelines. Most of these frameworks are based on deep neural networks which present a massive computational overhead on resource constrained platform like a mobile phone. In this paper, we propose several lightweight low-level modules which can be used to create a computationally low cost variant of a given baseline model. Recent works for efficient neural networks design have mainly focused on classification. However, low-level image processing falls under the 'image-to-image' translation genre which requires some additional computational modules not present in classification. This paper seeks to bridge this gap by designing generic efficient modules which can replace essential components used in contemporary deep learning based image restoration networks. We also present and analyse our results highlighting the drawbacks of applying depthwise separable convolutional kernel (a popular method for efficient classification network) for sub-pixel convolution based upsampling (a popular upsampling strategy for low-level vision applications). This shows that concepts from domain of classification cannot always be seamlessly integrated into 'image-to-image' translation tasks. We extensively validate our findings on three popular tasks of image inpainting, denoising and super-resolution. Our results show that proposed networks consistently output visually similar reconstructions compared to full capacity baselines with significant reduction of parameters, memory footprint and execution speeds on contemporary mobile devices. Codes are made available at https://github.com/avisekiit/TCSVT-LightWeight-CNNs † First two authors share equal contribution.

show abstract

A Disocclusion Inpainting Framework for Depth-Based View Synthesis

Cited by 30 publications

References 61 publications

Local Foreground Removal Disocclusion Filling Method for View Synthesis

Local Foreground Removal Disocclusion Filling Method for View Synthesis

Automatic Detection of Dispersed Defects in Resin Eyeglass Based on Machine Vision Technology

Lightweight Modules for Efficient Deep Learning Based Image Restoration

Contact Info

Product

Resources

About