100+ Times Faster Video Completion by Optical-Flow-Guided Variational Refinement

Bokov, Alexander; Vatolin, Dmitriy

doi:10.1109/icip.2018.8451683

Cited by 10 publications

(18 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A limitation of this work is that the estimation of the optical flow suffers from the presence of the scratches. Using a similar idea, but avoiding calculating the optical flow directly in the missing regions, several methods try to restore the motion field inside these missing regions by gradually propagating motion vectors [53], by sampling spatialtemporal motion patches [54,55], or by interpolating the missing motion [56,57].…”

Section: Video Inpaintingmentioning

confidence: 99%

“…This approach is inspired by the recently introduced method in Ref. [57], itself sharing ideas with the approach from Ref. [61] and yielding impressive speed gains.…”

Section: Video Inpaintingmentioning

confidence: 99%

“…A possible approach to optical flow inpainting is smooth interpolation, for instance, in the framework of a variational approach, by ignoring the data term and using only the smoothness term in the missing regions, as proposed in Refs. [56,57]. However, this approach leads to over-smoothed and unreliable optical flow.…”

Section: Motion Field Reconstructionmentioning

confidence: 99%

“…However, performing Poisson blending frame-by-frame may affect temporal consistency. To maintain it, we adopt the recent method of Bokov and Vatolin [57] which takes into account information from the previous frame. In this method, a regularizer penalizes discrepancies between the reconstructed colors and corresponding colors in the optical flowaligned previous frame.…”

Section: Motion Field Reconstructionmentioning

confidence: 99%

“…In this method, a regularizer penalizes discrepancies between the reconstructed colors and corresponding colors in the optical flowaligned previous frame. More specifically, given the colors of the current and previous inpainted frames I t (p), I t−1 (p), respectively, the refined Poissonblended image I(p) can be obtained by minimizing the discretized energy functional [57]:…”

Section: Motion Field Reconstructionmentioning

confidence: 99%

See 4 more Smart Citations

Object removal from complex videos using a few annotations

Almansa

Gousseau

et al. 2019

Comp. Visual Media

View full text Add to dashboard Cite

We present a system for the removal of objects from videos. As input, the system only needs a user to draw a few strokes on the first frame, roughly delimiting the objects to be removed. To the best of our knowledge, this is the first system allowing the semi-automatic removal of objects from videos with complex backgrounds. The key steps of our system are the following: after initialization, segmentation masks are first refined and then automatically propagated through the video. Missing regions are then synthesized using video inpainting techniques. Our system can deal with multiple, possibly crossing objects, with complex motions, and with dynamic textures. This results in a computational tool that can alleviate tedious manual operations for editing high-quality videos.

show abstract

Section: Video Inpaintingmentioning

confidence: 99%

“…This approach is inspired by the recently introduced method in Ref. [57], itself sharing ideas with the approach from Ref. [61] and yielding impressive speed gains.…”

Section: Video Inpaintingmentioning

confidence: 99%

Section: Motion Field Reconstructionmentioning

confidence: 99%

Section: Motion Field Reconstructionmentioning

confidence: 99%

Section: Motion Field Reconstructionmentioning

confidence: 99%

See 3 more Smart Citations

Object removal from complex videos using a few annotations

Almansa

Gousseau

et al. 2019

Comp. Visual Media

View full text Add to dashboard Cite

show abstract

Flow-edge Guided Video Completion

Gao

Saraf

Huang

et al. 2020

Lecture Notes in Computer Science

118

161

View full text Add to dashboard Cite

Optical flow correction network for repairing video

Huang

Luo

2023

SIViP

View full text Add to dashboard Cite

Some areas in the video may be defective due to storage medium damage or data transmission loss. The defective area can be repaired utilizing the optical flow since the optical flow represents the motion information of the pixel and contains the information redundancy properties of the video. However, it is a challenge for current algorithms to directly calculate the correct optical flow in the defective area. Therefore, an optical flow correction network (OFC-Net) is proposed to solve this problem. Firstly, as the basic framework, OFC-Net employs a lightweight U-shaped structure to gradually "repair" the broken optical flow map based on the known information in the defective neighborhood. And then the activation function that conforms to the property of optical flow is used to accelerate the model's convergence during training, while the simplified loss function is adopted to evaluate the similarity of the optical flow map from a global perspective to improve the prediction accuracy. As a result, the corrected optical flow output at the masked position can be obtained. Secondly, the optical flow corrected by OFC-Net can guide locating the pixels in nearby frames to repair the defective region. Optical flow correction and video repair experiments were conducted on MPI Sintel, DAVIS datasets, and real movie clips. The experimental results show that the corrected optical flow map is accurate without bringing artifacts under a wide range of scenarios and the arbitrary irregular mask shapes, which demonstrate the network's strong generalization capabilities. The average PSNR and SSIM of repaired video frames are greater than 33.67 and 0.986, respectively, outperforming the results from the traditional method PatchMatch and deep learning-based synthetic pixel methods.

show abstract

100+ Times Faster Video Completion by Optical-Flow-Guided Variational Refinement

Cited by 10 publications

References 18 publications

Object removal from complex videos using a few annotations

Object removal from complex videos using a few annotations

Flow-edge Guided Video Completion

Optical flow correction network for repairing video

Contact Info

Product

Resources

About