Robust Video Frame Interpolation With Exceptional Motion Map

Park, Minho; Kim, Hak Gu; Lee, Sangmin; Ro, Yong Man

doi:10.1109/tcsvt.2020.2981964

Cited by 27 publications

(7 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Liu et al [31] proposed a cycle consistency neural network in which the synthesized frames are asserted to be more reliable if they could be used to reconstruct the input frames accurately. Park et al [32] proposed a VFI method by considering the exceptional motion patterns. Lee et al [33] proposed a new warping module, namely adaptive collaboration of flows (AdaCoF), to estimate both kernel weights and offset vectors for each target pixel to synthesize the missing frame.…”

Section: Tsr For Videomentioning

confidence: 99%

Cuboid‐Net: A multi‐branch convolutional neural network for joint space‐time video super resolution

Fu,

Yuan,

et al. 2023

IET Image Processing

View full text Add to dashboard Cite

The demand for high‐resolution videos has been consistently rising across various domains, propelled by continuous advancements in societal. Nonetheless, limitations in imaging and economic factors often result in obtaining low‐resolution images. The currently available space‐time video super‐resolution methods often fail to fully exploit the information existing within the spatio‐temporal domain. To address this problem, the issue is tackled by conceptualizing the input low‐resolution video as a cuboid structure. An innovative methodology called “Cuboid‐Net”, which incorporates a multi‐branch convolutional neural network, is introduced. Cuboid‐Net is designed to collectively enhance the spatial and temporal resolutions of videos, enabling the extraction of rich and meaningful information across both spatial and temporal dimensions. Specifically, the input video is taken as a cuboid to generate different directional slices as input for different branches of the network. The proposed network contains four modules, that is, a multi‐branch‐based hybrid feature extraction module, a multi‐branch‐based reconstruction module, a first‐stage quality enhancement module, and a second‐stage cross frame quality enhancement module for interpolated frames only. Experimental results demonstrate that the proposed method is not only effective for spatial and temporal super‐resolution of video but also for spatial and angular super‐resolution of light field.

show abstract

Section: Tsr For Videomentioning

confidence: 99%

Cuboid‐Net: A multi‐branch convolutional neural network for joint space‐time video super resolution

Fu,

Yuan,

et al. 2023

IET Image Processing

View full text Add to dashboard Cite

show abstract

“…Interpolated frames, whose quality highly depends on the accuracy of the computationally expensive optical flow computation [32], typically suffer from motion boundaries and severe occlusions thus showing strong artifacts, even with state-of-the-art optical flow algorithms [33]. More recent promising works rely on neural networks to either predict convolution kernels for each pixel used to generate the interpolated frames [34] or leverage optical flow fields with exceptional motion maps [35]. However, these techniques involve a large number of convolutions, sometimes with large kernels (up to 41x41 for each pixel) to cope with large motion, thus making the computational demand unsuitable for real-rime use-cases.…”

Section: Motion Blur Rendering and Video Frame Interpolationmentioning

confidence: 99%

Quality-Driven Variable Frame-Rate for Green Video Coding in Broadcast Applications

Herrou

Bonnineau

Hamidouche

et al. 2021

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

The Digital Video Broadcasting (DVB) has proposed to introduce the Ultra-High Definition services in three phases: UHD-1 phase 1, UHD-1 phase 2 and UHD-2. The UHD-1 phase 2 specification includes several new features such as High Dynamic Range (HDR) and High Frame-Rate (HFR). It has been shown in several studies that HFR (+100 fps) enhances the perceptual quality and that this quality enhancement is contentdependent. On the other hand, HFR brings several challenges to the transmission chain including codec complexity increase and bit-rate overhead, which may delay or even prevent its deployment in the broadcast echo-system. In this paper, we propose a Variable Frame Rate (VFR) solution to determine the minimum (critical) frame-rate that preserves the perceived video quality of HFR video. The frame-rate determination is modeled as a 3-class classification problem which consists in dynamically and locally selecting one frame-rate among three: 30, 60 and 120 frames per second. Two random forests classifiers are trained with a ground truth carefully built by experts for this purpose. The subjective results conducted on ten HFR video contents, not included in the training set, clearly show the efficiency of the proposed solution enabling to locally determine the lowest possible frame-rate while preserving the quality of the HFR content. Moreover, our VFR solution enables significant bit-rate savings and complexity reductions at both encoder and decoder sides.

show abstract

“…The rapid pace of CNN research will certainly edge performance further ahead in the years to come. While this paper has been in review, this kind of exploration is beginning to emerge in the literature [54, 60, 61] with various CNN‐derived post‐processing strategies able to contribute another 1.3 dB [61] on to existing systems.…”

Section: Final Commentsmentioning

confidence: 99%

Motion‐based frame interpolation for film and television effects

Kokaram

Singh

Robinson

et al. 2020

IET Computer Vision

View full text Add to dashboard Cite

Frame interpolation is the process of synthesising a new frame in‐between existing frames in an image sequence. It has emerged as a key algorithmic module in motion picture effects. In the context of this special issue, this study provides a review of the technology used to create in‐between frames and presents a Bayesian framework that generalises frame interpolation algorithms using the concept of motion interpolation. Unlike existing literature in this area, the authors also compare performance using the top industrial toolkits used in the post production industry. They find that all successful techniques employ motion‐based interpolation, and the commercial version of the Bayesian approach performs best. Another goal of this study is to compare the performance gains with recent convolutional neural network (CNN) algorithms against the traditional explicit model‐based approaches. They find that CNNs do not clearly outperform the explicit motion‐based techniques, and require significant compute resources, but provide complementary improvements in certain types of sequences.

show abstract

Robust Video Frame Interpolation With Exceptional Motion Map

Cited by 27 publications

References 30 publications

Cuboid‐Net: A multi‐branch convolutional neural network for joint space‐time video super resolution

Cuboid‐Net: A multi‐branch convolutional neural network for joint space‐time video super resolution

Quality-Driven Variable Frame-Rate for Green Video Coding in Broadcast Applications

Motion‐based frame interpolation for film and television effects

Contact Info

Product

Resources

About