CDFI: Compression-Driven Network Design for Frame Interpolation

Abstract

DNN-based frame interpolation-that generates the intermediate frames given two consecutive frames-typically relies on heavy model architectures with a huge number of features, preventing them from being deployed on systems with limited resources, e.g., mobile devices. We propose a compression-driven network design for frame interpolation (CDFI), that leverages model pruning through sparsityinducing optimization to significantly reduce the model size while achieving superior performance. Concretely, we first compress the recently proposed AdaCoF model and show that a 10× compressed AdaCoF performs similarly as its original counterpart; then we further improve this compressed model by introducing a multi-resolution warping module, which boosts visual consistencies with multi-level details. As a consequence, we achieve a significant performance gain with only a quarter in size compared with the original AdaCoF. Moreover, our model performs favorably against other state-of-the-arts in a broad range of datasets. Finally, the proposed compression-driven framework is generic and can be easily transferred to other DNNbased frame interpolation algorithm. Our source code is available at https://github.com/tding1/CDFI. * Equal contribution. This work was done when Tianyu Ding was an intern at Applied Sciences Group, Microsoft.† Corresponding author.Recently, a large number of researches have been conducted in this area, especially those based on deep neural networks (DNN) for their promising results in motion esti-

Only Train Once: A One-Shot Neural Network Training And Pruning Framework

“…We argue that during model compression, OTO not only achieves parameter and FLOPs reduction, but also preserves the ability of capturing perceptual properties [75]. This is especially important in training and compressing models for many vision tasks, e.g., object detection [56,57], frame interpolation [2,15,53] and video synthesis [66,42]. We leave the application of OTO to broader tasks to future work.…”