Deep Flow-Guided Video Inpainting

Abstract

Video inpainting, which aims at filling in missing regions of a video, remains challenging due to the difficulty of preserving the precise spatial and temporal coherence of video contents. In this work we propose a novel flow-guided video inpainting approach. Rather than filling in the RGB pixels of each frame directly, we consider video inpainting as a pixel propagation problem. We first synthesize a spatially and temporally coherent optical flow field across video frames using a newly designed Deep Flow Completion network. Then the synthesized flow field is used to guide the propagation of pixels to fill up the missing regions in the video. Specifically, the Deep Flow Completion network follows a coarse-to-fine refinement to complete the flow fields, while their quality is further improved by hard flow example mining. Following the guide of the completed flow, the missing video regions can be filled up precisely. Our method is evaluated on DAVIS and YouTube-VOS datasets qualitatively and quantitatively, achieving the state-of-the-art performance in terms of inpainting quality and speed. The project page is available at https: //nbei.github.io/video-inpainting.html

Sequential and Patch Analyses for Object Removal Video Forgery Detection and Localization

“…Object removal video forgery is achieved by using inpainting algorithms [33]- [35]. The following works are proposed to detect object removal video forgery [36]- [44].…”

“…In other words, object-based forgery is performed in the middle of frames, e.g., a walking person is removed before leaving a video scene, so this person is seen for a couple of seconds and suddenly disappeared from the video scene. Hence, we use SYSU-OBJFORG data set to generate realistic object removal forged videos by using two recent inpainting algorithms [33], [35]. Fig.7 shows three examples of object removal forgery from the data set.…”

“…We measure the performance at pixel level by defining T P as the correctly detected forged pixels, F P as original pixels that have been incorrectly detected as forged, and F N as falsely missed forged pixels. Then, we compute P recision, Recall, F 1, and IoU as in (33), (34), (35), and (36) respectively.…”

An Internal Learning Approach to Video Inpainting

“…Most existing video inpainting methods build on patch-based synthesis with spatial-temporal matching [16,27,33,44] or explicit motion estimation and tracking [1,6,8,9]. Very recently, deep convolutional networks have been used to directly inpaint holes in videos and achieve promising results [24,41,45], leveraging large external video corpus for training along with specialized recurrent frameworks to model spatial-temporal coherence. Different from their works, we explore the orthogonal direction of learning-based video inpainting by investigating an internal (within-video) learning approach.…”

Figure, Ground and the Notion of Equilibria in the Work of Gilbert Simondon and Gestalt theory

“…etwa Li, Gauci & Groß, 2016) bewertet, siehe Goodfellow et al (2014). 20 Anschauliche Beispiele für maschinelle Mustererkennung und deren produktive Umsetzung sind, neben der beeindruckenden Mustererkennung zur Bild-und Videobearbeitung (Xu et al, 2019) z.B. maschinell generierte Bilder, wie sie von Klingemann n.D., Barrat n.D., Bethge n.D., Valenzuela n.D. gezeigt werden, sowie aktuell die Ergebnisse von BigGAN (vgl.…”