Towards fast, generic video inpainting

Newson, Alasdair; Almansa, Andrés; Fradet, Matthieu; Gousseau, Yann; Pérez, Patrick

doi:10.1145/2534008.2534019

Cited by 28 publications

(21 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For the photographs, we also compare the output of Guidefill with four other inpainting methods -coherence transport [7,27], the variational exemplar-based methods nl-means and nl-Poisson from Arias et al [3], and Photoshop's Content-Aware fill. For the movie, we compare with the exemplar-based video inpainting method of Newson et al [32,31]. However, generating "bystander-aware" versions of all of these methods would have been a significant undertaking, so we arrived at a compromise.…”

Section: D Conversion Examplesmentioning

confidence: 99%

Guidefill: GPU Accelerated, Artist Guided Geometric Inpainting for 3D Conversion of Film

Hocking¹,

MacKenzie²,

Schönlieb³

2017

SIAM J. Imaging Sci.

View full text Add to dashboard Cite

The conversion of traditional film into stereo 3D has become an important problem in the past decade. One of the main bottlenecks is a disocclusion step, which in commercial 3D conversion is usually done by teams of artists armed with a toolbox of inpainting algorithms. A current difficulty in this is that most available algorithms are either too slow for interactive use, or provide no intuitive means for users to tweak the output.In this paper we present a new fast inpainting algorithm based on transporting along automatically detected splines, which the user may edit. Our algorithm is implemented on the GPU and fills the inpainting domain in successive shells that adapt their shape on the fly. In order to allocate GPU resources as efficiently as possible, we propose a parallel algorithm to track the inpainting interface as it evolves, ensuring that no resources are wasted on pixels that are not currently being worked on. Theoretical analysis of the time and processor complexity of our algorithm without and with tracking (as well as numerous numerical experiments) demonstrate the merits of the latter.Our transport mechanism is similar to the one used in coherence transport [7,27], but improves upon it by correcting a "kinking" phenomenon whereby extrapolated isophotes may bend at the boundary of the inpainting domain. Theoretical results explaining this phenomena and its resolution are presented.Although our method ignores texture, in many cases this is not a problem due to the thin inpainting domains in 3D conversion. Experimental results show that our method can achieve a visual quality that is competitive with the state-of-the-art while maintaining interactive speeds and providing the user with an intuitive interface to tweak the results.

show abstract

Section: D Conversion Examplesmentioning

confidence: 99%

Guidefill: GPU Accelerated, Artist Guided Geometric Inpainting for 3D Conversion of Film

Hocking¹,

MacKenzie²,

Schönlieb³

2017

SIAM J. Imaging Sci.

View full text Add to dashboard Cite

show abstract

“…3d versions of PatchMatch have been already proposed, recently, like in [15] for stereo matching, and in [16] for video inpainting. In both cases, however, the algorithm works on the RGB values of the patch.…”

Section: Related Workmentioning

confidence: 99%

Video forgery detection and localization based on 3D patchmatch

D'Amiano¹,

Cozzolino²,

Poggi³

et al. 2015

2015 IEEE International Conference on Multimedia &Amp; Expo Workshops (ICMEW)

View full text Add to dashboard Cite

A new algorithm is proposed for the detection and accurate localization of copy-move video forgeries. Major ingredients of the proposed algorithm are i) noise-resilient rotation-invariant features, ii) dense-field matching over the whole video by means of a suitably modified fast algorithm for approximate nearest neighbor search, iii) ad hoc fast post-processing for forgery detection and false alarm removal. Experiments carried out on a dataset publicly available for this task show promising results, suggesting also a number of directions for future research.

show abstract

“…A patch-based method [6,8,10,29] completes missing regions by sampling and pasting patches from known regions or other source images. This kind of approach has been extended to the temporal domain for video inpainting [21,22,27]. Strobel et al [30] and Huang et al [12] further estimate the motion field in the missing regions to address the temporal consistency problem.…”

Section: Related Workmentioning

confidence: 99%

Deep Flow-Guided Video Inpainting

Zhou

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

244

196

View full text Add to dashboard Cite

Video inpainting, which aims at filling in missing regions of a video, remains challenging due to the difficulty of preserving the precise spatial and temporal coherence of video contents. In this work we propose a novel flow-guided video inpainting approach. Rather than filling in the RGB pixels of each frame directly, we consider video inpainting as a pixel propagation problem. We first synthesize a spatially and temporally coherent optical flow field across video frames using a newly designed Deep Flow Completion network. Then the synthesized flow field is used to guide the propagation of pixels to fill up the missing regions in the video. Specifically, the Deep Flow Completion network follows a coarse-to-fine refinement to complete the flow fields, while their quality is further improved by hard flow example mining. Following the guide of the completed flow, the missing video regions can be filled up precisely. Our method is evaluated on DAVIS and YouTube-VOS datasets qualitatively and quantitatively, achieving the state-of-the-art performance in terms of inpainting quality and speed. The project page is available at https: //nbei.github.io/video-inpainting.html

show abstract

Towards fast, generic video inpainting

Cited by 28 publications

References 18 publications

Guidefill: GPU Accelerated, Artist Guided Geometric Inpainting for 3D Conversion of Film

Guidefill: GPU Accelerated, Artist Guided Geometric Inpainting for 3D Conversion of Film

Video forgery detection and localization based on 3D patchmatch

Deep Flow-Guided Video Inpainting

Contact Info

Product

Resources

About