Motion-consistent video inpainting

Le, Thuc Trinh; Almansa, Andrés; Gousseau, Yann; Masnou, Simon

doi:10.1109/icip.2017.8296651

Cited by 28 publications

(15 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Spatially localised changes in video footage, such as face swapping, can change the entire context of a news story or film and can have repercussions for the people portrayed. Already, videos which have been cleverly edited to change the context of what was said by influential people have gone viral 1 . If that can be done with unsophisticated editing techniques, it is worth considering what more could be achieved with modern techniques.…”

Section: Accepted Manuscriptmentioning

confidence: 99%

See 1 more Smart Citation

A review of digital video tampering: From simple editing to full synthesis

Johnston

Elyan

2019

Digital Investigation

View full text Add to dashboard Cite

Video tampering methods have witnessed considerable progress in recent years. This is partly due to the rapid development of advanced deep learning methods, and also due to the large volume of video footage that is now in the public domain. Historically, convincing video tampering has been too labour intensive to achieve on a large scale. However, recent developments in deep learning-based methods have made it possible not only to produce convincing forged video but also to fully synthesize video content. Such advancements provide new means to improve visual content itself, but at the same time, they raise new challenges for state-of-the-art tampering detection methods. Video tampering detection has been an active field of research for some time, with periodic reviews of the subject. However, little attention has been paid to video tampering techniques themselves. This paper provides an objective and in-depth examination of current techniques related to digital video manipulation. We thoroughly examine their development, and show how current evaluation techniques provide opportunities for the advancement of video tampering detection. A critical and extensive review of photo-realistic video synthesis is provided with emphasis on deep learning-based methods. Existing tampered video datasets are also qualitatively reviewed and critically discussed. Finally, conclusions are drawn upon an exhaustive and thorough review of tampering methods with

show abstract

Section: Accepted Manuscriptmentioning

confidence: 99%

“…The synthesis of convincing fake video content has increased recently due to the development of intelligent models [1,2,3]. Selective modification of image content has been possible for some years, but the application of similar techniques to video has been too labour intensive to see mass use.…”

Section: Introductionmentioning

confidence: 99%

A review of digital video tampering: From simple editing to full synthesis

Johnston

Elyan

2019

Digital Investigation

View full text Add to dashboard Cite

show abstract

“…Finally, recent methods consider a mixture of patches following a spatial non-local search as in [1]. This patch-based approach for inpainting was also extended to videos in [22] and [19] with 3D patches to include some temporal similarity in patch comparisons. If these models yield better results in the recovery of texture, they are however highly dependent on the initial filling of the area, so as not to remain blocked in a local minimum for the solution, which usually fails at reconstructing regular structures.…”

Section: Related Workmentioning

confidence: 99%

“…Here the search for optimal patches is no longer carried out in a spatial neighbourhood around the defect, but in a temporal neighbourhood (in the previous frame u b and the next frame u f ). While in [22] and [19] the 3D patch search is not limited in time distance, here we focus only on 2D patches in the backward and forward frames. In the current frame u, the central pixel x of the patch p u (x) concerned by the search of the optimal patch in the neighbour frames u b and u f is in the area O which is O expanded by half a patch width, in order to propagate patches containing sufficient healthy data:…”

Section: Textural Reconstruction Energymentioning

confidence: 99%

Alternate Structural-Textural Video Inpainting for Spot Defects Correction in Movies

Renaudeau

Lauze

Pierre

et al. 2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

We propose a new video inpainting model for movies restoration application. Our model combines structural reconstruction with a diffusion-based method and textural reconstruction with a patch-based method. Both proposed energies (one for each method) are alternatively minimized in order to preserve the overall structure while adding textural refinement. While the structural reconstruction is obtained jointly with optical flow computation with several proximal approaches, the textural reconstruction is processed by a variational non-local approach. Preliminary results on different Middlebury frames show quality improvement in the reconstruction.

show abstract

“…Early patch-based inpainting methods [10,11,39] divide images into small patches and recover the masked region by pasting the most similar patch somewhere in the image/video. These methods could generate authentic results but they are usually very time-consuming due to the complexity of neighbor-finding algorithms [26]. In addition, patch-based methods assume there is a reference for the missing part and often fail to recover non-repetitive and complex region (e.g, they cannot recover a missing face well [28]).…”

Section: Introductionmentioning

confidence: 99%

VORNet: Spatio-Temporally Consistent Video Inpainting for Object Removal

Chang

Liu

Hsu

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

View full text Add to dashboard Cite

c) Ours (b) Image-based (a) Input video Time order Figure 1. Video object removal results. (a) The input video and its foreground bounding boxes to remove, marked in red. (b) State-of-the-art image-based inpainting model by Yu et al. [50]. (c) Our results.Applying image-based algorithms on the video inpainting task often leads to temporal inconsistency, where the content is different in each frame, e.g., the windows are missing in the last frame of (b). Our deep learning based architecture could improve both the spatial and temporal consistency of image-based inpainting models. Best viewed with color and zoom-in. See http://bit.ly/2GkW9Kr for the video. AbstractVideo object removal is a challenging task in video processing that often requires massive human efforts. Given the mask of the foreground object in each frame, the goal is to complete (inpaint) the object region and generate a video without the target object. While recently deep learning based methods have achieved great success on the image inpainting task, they often lead to inconsistent results between frames when applied to videos. In this work, we propose a novel learning-based Video Object Removal Network (VORNet) to solve the video object removal task in a spatio-temporally consistent manner, by combining the optical flow warping and image-based inpainting model. Experiments are done on our Synthesized Video Object Removal (SVOR) dataset based on the YouTube-VOS video segmentation dataset, and both the objective and subjective evaluation demonstrate that our VORNet generates more spatially and temporally consistent videos compared with existing methods.

show abstract

Motion-consistent video inpainting

Cited by 28 publications

References 22 publications

A review of digital video tampering: From simple editing to full synthesis

A review of digital video tampering: From simple editing to full synthesis

Alternate Structural-Textural Video Inpainting for Spot Defects Correction in Movies

VORNet: Spatio-Temporally Consistent Video Inpainting for Object Removal

Contact Info

Product

Resources

About