Revisiting RCAN: Improved Training for Image Super-Resolution

Abstract

Image super-resolution (SR) is a fast-moving field with novel architectures attracting the spotlight. However, most SR models were optimized with dated training strategies. In this work, we revisit the popular RCAN model and examine the effect of different training options in SR. Surprisingly (or perhaps as expected), we show that RCAN can outperform or match nearly all the CNN-based SR architectures published after RCAN on standard benchmarks with a proper training strategy and minimal architecture change. Besides, although RCAN is a very large SR architecture with more than four hundred convolutional layers, we draw a notable conclusion that underfitting is still the main problem restricting the model capability instead of overfitting. We observe supportive evidence that increasing training iterations clearly improves the model performance while applying regularization techniques generally degrades the predictions. We denote our simply revised RCAN as RCAN-it and recommend practitioners to use it as baselines for future research. Code is publicly available at https://github.com/zudi-lin/rcan-it.

Activating More Pixels in Image Super-Resolution Transformer

“…Tab. 6 shows the quantitative comparison of our approach and the state-of-the-art methods: EDSR [30], RCAN [65], SAN [7], IGNN [69], HAN [41], NLSN [40], RCAN-it [32], as well as approaches using Ima-geNet pre-training, i.e., IPT [5] and EDT [26]. We can see that our approach outperforms the other methods significantly on all benchmark datasets.…”

“…We use DF2K dataset (DIV2K [31]+Flicker2K [47]) as the original training dataset, following the latest publications [29,32]. When utilizing pre-training, we adopt ImageNet [8] following [5,26].…”

NTIRE 2021 Challenge on Quality Enhancement of Compressed Video: Methods and Results

“…To be specific, since the modules of feature extracting and upsampling are far weaker than that of feature propagation, they increase the number of residual blocks in these two modules. Besides, enhanced activation functions (e.g., SiLU [17]) are proven to be effective according to [40]. For the sake of training efficacy, they replace Leaky ReLU in BasicVSR++ with PReLU [22] and verify its effectiveness.…”

“…Figure 3 shows the pipeline of the proposed method. Inspired by [40], an enlarged model combined with proper training strategies is expected to have noticeable improvements over the baseline. On the one hand, they perform two modifications on BasicVSR++ to improve its capacity.…”

“…Given the success of the BasicVSR++ model in NTIRE 2021 [10], they focus on the training strategies using this model. Based on the recommendations in [32,41,43], they applied several strategies including: adding spatial dropout [58] before the last layer of the model, using large batch size (240) and increasing the batch volume focused mostly on an increased number of frames (from 75 to 120). The major difficulty is the memory requirements to run training on large batches and for this purpose they use the automatic mixed-precision library APEX 3 from NVIDIA.…”