CrossNet++: Cross-Scale Large-Parallax Warping for Reference-Based Super-Resolution

Yang, Tan; Zheng, Haitian; Zhu, Yinheng; Yuan, Xiaoyun; Lin, Xing; Brady, David J.; Fang, Lü

doi:10.1109/tpami.2020.2997007

Cited by 32 publications

(21 citation statements)

References 59 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Zheng et al [41] proposed a CrossNet and used optical flow to align input and reference images. Subsequently, they proposed CrossNet++ [47] to tackle the RefSR problem with a significant resolution gap (8×). Zhang et al [42] proposed a SRNTT network to adaptively transfer the texture from reference images to the target image according to their textural similarity.…”

Section: B Multi-image Srmentioning

confidence: 99%

Learning Parallax Attention for Stereo Image Super-Resolution

Wang

Liang³

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

257

228

View full text Add to dashboard Cite

Stereo image pairs can be used to improve the performance of super-resolution (SR) since additional information is provided from a second viewpoint. However, it is challenging to incorporate this information for SR since disparities between stereo images vary significantly. In this paper, we propose a parallax-attention stereo superresolution network (PASSRnet) to integrate the information from a stereo image pair for SR. Specifically, we introduce a parallax-attention mechanism with a global receptive field along the epipolar line to handle different stereo images with large disparity variations. We also propose a new and the largest dataset for stereo image SR (namely, Flickr1024). Extensive experiments demonstrate that the parallax-attention mechanism can capture correspondence between stereo images to improve SR performance with a small computational and memory cost. Comparative results show that our PASSRnet achieves the state-of-the-art performance on the Middlebury, KITTI 2012 and KITTI 2015 datasets.

show abstract

Section: B Multi-image Srmentioning

confidence: 99%

Learning Parallax Attention for Stereo Image Super-Resolution

Wang

Liang³

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

257

228

View full text Add to dashboard Cite

show abstract

“…In addition, Zheng et al [30] propose a cross-resolution patch matching and synthesis scheme for RefSR. To improve RefSR, more recent works [32,17] utilize cross-scale optical flow networks and a warping-synthesis framework to perform RefSR prediction. Aslo there is work [29] leveraging patch-based correlation to perform non-rigid feature swapping for reference-based SR.…”

Section: Reference-based Super-resolutionmentioning

confidence: 99%

“…1). While numerous methods [3,31,32,17] have been proposed for reference-based super-resolution (RefSR), some of them even have been applied in practice such as giga-pixel imaging [5,27,28] and light-field reconstruction [3,22]. The RefSR task for videos remains challenging, due to the following two factors: 1) the large parallax and resolution gap (e.g., 4×) makes it difficult to transfer details from HR frame to LR ones; 2) the potential large temporal gap between the HR and LR frames can lead to significant viewpoint drift among frames, therefore makes the regarding correspondence estimation error-prone.…”

Section: Introductionmentioning

confidence: 99%

“…To better align frames across different resolutions, recent works [32,17] for RefSR apply a flow estimator from [8] to estimate the correspondence. Though such methods can be used in RefVSR by dividing the RefVSR task into multiple separated RefSR tasks, they discard the important temporal information contained in video sequence.…”

Section: Introductionmentioning

confidence: 99%

“…To overcome the aforementioned difficulties, we propose an end-to-end neural network, EFENet 1 , for RefVSR. EFENet takes advantage of the state-of-theart RefSR method CrossNet++ [17], but, more crucially, introduces a novel flow refinement module to improve the furthest frame alignment with all the prior ones, allowing the usage of temporal information for long range video synthesis on top of the powerful correspondence estimation module. In the end, EFENet decodes the refined flow to output super-resolved video sequences.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations