Frame rate up conversion via image fusion based on variational approach

Lee, Won Hee; Choi, Yunjun; Choi, Kyuha; Beom, Jong

doi:10.1109/icip.2010.5654075

Cited by 7 publications

(8 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Video Frame Interpolation. Existing methods for frame interpolation generally utilize optical flow to process motion information [18,1,2,9,15,37,23] or use kernelbased models [19,24,25]. As a pioneer of learning-based methods, Long et al [19] train a generic convolutional neural network to synthesize the intermediate frame directly.…”

Section: Related Workmentioning

confidence: 99%

Blurry Video Frame Interpolation

Shen

Bao

Zhai

et al. 2020

Preprint

View full text Add to dashboard Cite

Existing works reduce motion blur and up-convert frame rate through two separate ways, including frame deblurring and frame interpolation. However, few studies have approached the joint video enhancement problem, namely synthesizing high-frame-rate clear results from low-framerate blurry inputs. In this paper, we propose a blurry video frame interpolation method to reduce motion blur and upconvert frame rate simultaneously. Specifically, we develop a pyramid module to cyclically synthesize clear intermediate frames. The pyramid module features adjustable spatial receptive field and temporal scope, thus contributing to controllable computational complexity and restoration ability. Besides, we propose an inter-pyramid recurrent module to connect sequential models to exploit the temporal relationship. The pyramid module integrates a recurrent module, thus can iteratively synthesize temporally smooth results without significantly increasing the model size. Extensive experimental results demonstrate that our method performs favorably against state-of-the-art methods. The source code and pre-trained model are available at https://github.com/laomao0/BIN .

show abstract

Section: Related Workmentioning

confidence: 99%

Blurry Video Frame Interpolation

Shen

Bao

Zhai

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…It is difficult to calculate the total number of operations due to the cumbersome formulas (11) and uncertain parameters S and N c , and therefore we use big O to represent the total computational complexity of proposed BME algorithm. Because the maximums of S and N c are N B (= N 1 N 2 /BK 2 ) and 9 respectively, and it can be seen from (11) …”

Section: Analysis Of Computational Complexitymentioning

confidence: 99%

“…Focusing on the better quality of interpolated frames, Ref. [11] uses BME with various block size to compute multiple interpolated frames, and then introduces the image fusion algorithm to generate the final inCopyright c 2016 The Institute of Electronics, Information and Communication Engineers terpolated frame. The drawback of this method is its high computational complexity when compared with the traditional method based on the generation of a single MVF.…”

Section: Introductionmentioning

confidence: 99%

“…The study of scan order in full search and the pipelining of MV post-processing are presented in [13] to improve the accuracy of BME. The success of above-mentioned methods mostly depends on the supplements of MV post-processing and MCI algorithms, such as PMVS, PAMC and IPHI in [10], image fusion method in [11], etc. Then, are there some simple and classic methods to modify the internal mechanism of BME for obtaining the more better performance?…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Wavelet Pyramid Based Multi-Resolution Bilateral Motion Estimation for Frame Rate Up-Conversion

Liu

Chen

et al. 2016

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

SUMMARYThe conventional bilateral motion estimation (BME) for motion-compensated frame rate up-conversion (MC-FRUC) can avoid the problem of overlapped areas and holes but usually results in lots of inaccurate motion vectors (MVs) since 1) the MV of an object between the previous and following frames is more likely to have no temporal symmetry with respect to the target block of the interpolated frame and 2) the repetitive patterns existing in video frame lead to the problem of mismatch due to the lack of the interpolated block. In this paper, a new BME algorithm with a low computational complexity is proposed to resolve the above problems. The proposed algorithm incorporates multi-resolution search into BME, since it can easily utilize the MV consistency between two adjacent pyramid levels and spatial neighboring MVs to correct the inaccurate MVs resulting from no temporal symmetry while guaranteeing low computational cost. Besides, the multi-resolution search uses the fast wavelet transform to construct the wavelet pyramid, which not only can guarantee low computational complexity but also can reserve the high-frequency components of image at each level while sub-sampling. The high-frequency components are used to regularize the traditional block matching criterion for reducing the probability of mismatch in BME. Experiments show that the proposed algorithm can significantly improve both the objective and subjective quality of the interpolated frame with low computational complexity, and provide the better performance than the existing BME algorithms.

show abstract

“…The methods of frame interpolation use opticalflow fields to generate inter-frame images. Several methods of frame-rate up-conversion for an image sequence using the optical flow field have been proposed (Werlberger et al, 2011;Lee et al, 2010). In this conversion, although for the generation of subframe images, the frame-rate up-conversion of an image sequence is a fundamental requirement, our objective is the up-conversion of the optical flow field sequence.…”

Section: Introductionmentioning

confidence: 99%

Simultaneous Frame-rate Up-conversion of Image and Optical Flow Sequences

Inagaki

Itoh

Imiya

2015

Proceedings of the 10th International Conference on Computer Vision Theory and Applications

View full text Add to dashboard Cite

We develop a variational method for the frame-rate up-conversion of optical flow fields, in which we combine motion coherency in an image sequence and the smoothness of the temporal flow field. Since optical flow vectors define the motion of each point in an image, we can construct interframe images from low frame-rate image sequences using flow field vectors. The algorithm produces both interframe images and optical flow fields from a set of successive images in a sequence.

show abstract

Frame rate up conversion via image fusion based on variational approach

Cited by 7 publications

References 7 publications

Blurry Video Frame Interpolation

Blurry Video Frame Interpolation

Wavelet Pyramid Based Multi-Resolution Bilateral Motion Estimation for Frame Rate Up-Conversion

Simultaneous Frame-rate Up-conversion of Image and Optical Flow Sequences

Contact Info

Product

Resources

About