Examplar-based inpainting based on local geometry

Meur, Olivier Le; Gautier, Josselin; Guillemot, Christine

doi:10.1109/icip.2011.6116441

Cited by 94 publications

(92 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Clearly, our method provides much better inpainting results then [11]. This shows that inpainting of depth map itself is not trivial and cannot be done simply, as claimed in [6], [7], [8].…”

Section: Subjective Resultsmentioning

confidence: 71%

“…Numerous subsequent works [6], [7], [8], [9], [10] kept the template matching framework in [11] but modified the definition of the priority term (used to determine the order in which missing pixel patches should be filled) and/or the criteria for template matching, using available depth information. The underlying assumption for the majority of these works ( [6], [7], [8]), however, is that a complete and good-quality depth map at the target virtual view is available, or can be easily pre-computed a priori, for the said computation of the priority term and/or matching criteria.…”

Section: Related Workmentioning

confidence: 99%

“…There are in general two classes of inpainting algorithms: partial differential equations (PDEs)-based schemes like [12] and exemplar-based (template matching) schemes [11], [6], [7], [8], [9], [10]. It is known that PDEs-based schemes do not handle large disocclusion holes well-common if the spacing between neighboring cameras is large, or if the virtual view image is synthesized using one texture/depth map pair of a single camera view.…”

Section: Related Workmentioning

confidence: 99%

“…Though a similar template matching framework introduced in [11] is used to copy texture pixels from the known region to the unknown region, we derive a new priority term to order filling of pixel patches using available partial depth information. Further, unlike [6], [7], [8] whose inpainting performance depends heavily on the availability of a complete and good-quality depth map in the virtual view for texture inpainting, in JTDI a more realistic DIBR view synthesis scenario is assumed where depth pixels in the disoccluded regions are also missing and challenging to complete. So a joint inpainting algorithm is required to carefully fill in missing pixels in both texture and depth maps.…”

Section: Introductionmentioning

confidence: 99%

“…The hole with no corresponding pixels in the reference view is commonly called a disocclusion hole. Devising a strategy to properly fill in missing pixels in a disocclusion hole-a process called inpainting or image completion in the literature [6], [7], [8], [9], [10]-is paramount in constructing a visually pleasing virtual viewpoint image.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Joint texture-depth pixel inpainting of disocclusion holes in virtual view synthesis

Reel

Cheung

Wong

et al. 2013

2013 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference

View full text Add to dashboard Cite

Abstract-Transmitting texture and depth maps from one or more reference views enables a user to freely choose virtual viewpoints from which to synthesize images for observation via depth-image-based rendering (DIBR). In each DIBR-synthesized image, however, there remain disocclusion holes with missing pixels corresponding to spatial regions occluded from view in the reference images. To complete these holes, unlike previous schemes that rely heavily (and unrealistically) on the availability of a high-quality depth map in the virtual view for inpainting of the corresponding texture map, in this paper a new Joint TextureDepth Inpainting (JTDI) algorithm is proposed that simultaneously fill in missing texture and depth pixels. Specifically, we first use available partial depth information to compute priority terms to identify the next target pixel patch in a disocclusion hole for inpainting. Then, after identifying the best-matched texture patch in the known pixel region via template matching for texture inpainting, the variance of the corresponding depth patch is copied to the target depth patch for depth inpainting. Experimental results show that JTDI outperforms two previous inpainting schemes that either does not use available depth information during inpainting, or depends on the availability of a good depth map at the virtual view for good inpainting performance.

show abstract

Section: Subjective Resultsmentioning

confidence: 71%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Joint texture-depth pixel inpainting of disocclusion holes in virtual view synthesis

Reel

Cheung

Wong

et al. 2013

2013 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference

View full text Add to dashboard Cite

show abstract

Shift-Net: Image Inpainting via Deep Feature Rearrangement

Yan

et al. 2018

Lecture Notes in Computer Science

426

335

View full text Add to dashboard Cite

Deep convolutional networks (CNNs) have exhibited their potential in image inpainting for producing plausible results. However, in most existing methods, e.g., context encoder, the missing parts are predicted by propagating the surrounding convolutional features through a fully connected layer, which intends to produce semantically plausible but blurry result. In this paper, we introduce a special shift-connection layer to the U-Net architecture, namely Shift-Net, for filling in missing regions of any shape with sharp structures and fine-detailed textures. To this end, the encoder feature of the known region is shifted to serve as an estimation of the missing parts. A guidance loss is introduced on decoder feature to minimize the distance between the decoder feature after fully connected layer and the ground-truth encoder feature of the missing parts. With such constraint, the decoder feature in missing region can be used to guide the shift of encoder feature in known region. An end-to-end learning algorithm is further developed to train the Shift-Net. Experiments on the Paris StreetView and Places datasets demonstrate the efficiency and effectiveness of our Shift-Net in producing sharper, fine-detailed, and visually plausible results. The codes and pre-trained models are available at https://github.com/Zhaoyi-Yan/Shift-Net.

show abstract

Image Inpainting with Learnable Feature Imputation

Hukkelås

Lindseth

Mester

2021

Lecture Notes in Computer Science

View full text Add to dashboard Cite

A regular convolution layer applying a filter in the same way over known and unknown areas causes visual artifacts in the inpainted image. Several studies address this issue with feature re-normalization on the output of the convolution. However, these models use a significant amount of learnable parameters for feature re-normalization [ 41 , 48 ], or assume a binary representation of the certainty of an output [ 11 , 26 ]. We propose (layer-wise) feature imputation of the missing input values to a convolution. In contrast to learned feature re-normalization [ 41 , 48 ], our method is efficient and introduces a minimal number of parameters. Furthermore, we propose a revised gradient penalty for image inpainting, and a novel GAN architecture trained exclusively on adversarial loss. Our quantitative evaluation on the FDF dataset reflects that our revised gradient penalty and alternative convolution improves generated image quality significantly. We present comparisons on CelebA-HQ and Places2 to current state-of-the-art to validate our model. (Code is available at: github.com/hukkelas/DeepPrivacy . Supplementary material can be downloaded from: folk.ntnu.no/haakohu/GCPR_supplementary.pdf ) Electronic supplementary material The online version of this chapter (10.1007/978-3-030-71278-5_28) contains supplementary material, which is available to authorized users.

show abstract

Examplar-based inpainting based on local geometry

Cited by 94 publications

References 11 publications

Joint texture-depth pixel inpainting of disocclusion holes in virtual view synthesis

Joint texture-depth pixel inpainting of disocclusion holes in virtual view synthesis

Shift-Net: Image Inpainting via Deep Feature Rearrangement

Image Inpainting with Learnable Feature Imputation

Contact Info

Product

Resources

About