Edge-Aware Network for Flow-Based Video Frame Interpolation

Zhao, Lei; Li, Xuelong

doi:10.1109/tnnls.2022.3178281

Cited by 6 publications

(1 citation statement)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, our motion context joint refinement based network runs 11× faster than multi-branch based CDFI [18], demonstrating the efficiency of our approach. For SSIM, our PMCRNet behaves better than recent methods EDSC [27] and EA-Net [28] on Vimeo90K, and achieves the same best accuracy as CAIN [7] on UCF101 but containing 6.9× less parameters. As for Middlebury benchmark, we also rank 1st on both PSNR and SSIM, while only fall behind CDFI [18] by 0.02 on the IE metric.…”

Section: B Comparison To the State-of-the-artsmentioning

confidence: 86%

Progressive Motion Context Refine Network for Efficient Video Frame Interpolation

Kong

Liu

Yang

2022

IEEE Signal Process. Lett.

View full text Add to dashboard Cite

Recently, flow-based frame interpolation methods have achieved great success by first modeling optical flow between target and input frames, and then building synthesis network for target frame generation. However, above cascaded architecture can lead to large model size and inference delay, hindering them from mobile and real-time applications. To solve this problem, we propose a novel Progressive Motion Context Refine Network (PMCRNet) to predict motion fields and image context jointly for higher efficiency. Different from others that directly synthesize target frame from deep feature, we explore to simplify frame interpolation task by borrowing existing texture from adjacent input frames, which means that decoder in each pyramid level of our PMCRNet only needs to update easier intermediate optical flow, occlusion merge mask and image residual. Moreover, we introduce a new annealed multi-scale reconstruction loss to better guide the learning process of this efficient PMCRNet. Experiments on multiple benchmarks show that proposed approaches not only achieve favorable quantitative and qualitative results but also reduces current model size and running time significantly.

show abstract