Light Field Image Super-Resolution Using Deformable Convolution

Wang, Yingqian; Yang, Jungang; Wang, Longguang; Ying, Xinyi; Wu, Tianhao; An, Wei; Guo, Yulan

doi:10.1109/tip.2020.3042059

Cited by 122 publications

(128 citation statements)

References 85 publications

(139 reference statements)

Supporting

Mentioning

128

Contrasting

Order By: Relevance

“…3. Most LFSR methods [35]- [38] adopt the implicit feature alignment methods (shown in Fig. 3 (a)) which directly applies a CNN network to learn the complementary information of the reference feature and target feature.…”

Section: Discussionmentioning

confidence: 99%

“…However, this approach is not efficient because it doesn't explicitly model the difference between SAIs' features, which hinders the final LFSR performance. Wang et al [38] utilizes the deformable convolution to align the reference feature and target feature, as shown in Fig. 3…”

Section: Discussionmentioning

confidence: 99%

“…Following [20], [36], [38], we convert the input LF image from the RGB space to the YCbCr space. Given an LF image, only the Y channel of the image is super-resolved, and the bicubic interpolation is used to process the Cb and Cr channel of the images.…”

Section: Proposed Methodsmentioning

confidence: 99%

“…Recently, the research interests of LFSR are mainly about how to utilize the different views of sub-aperture images efficiently. They employed part [35], [36] or all [37], [38] of sub-aperture images to learn the complementary information of each other for final LFSR. Al-though the above methods achieve satisfactory performance, the complementary information of different sub-aperture images is not explicitly modeled to improve super-resolution results.…”

Section: A Light Field Image Super-resolutionmentioning

confidence: 99%

See 3 more Smart Citations

Light Field Image Super-Resolution via Mutual Attention Guidance

Wang

2021

IEEE Access

View full text Add to dashboard Cite

Deep learning-based methods have prompted light field image super-resolution to achieve significant progress. However, most of them ignore aligning different sub-aperture features of light field image before aggregation, resulting in sub-optimal super-resolution results. We aim to propose an efficient feature alignment method for sub-aperture feature aggregation. To this end, we develop a mutual attention mechanism for sub-aperture feature alignment and propose a mutual attention guidance block (MAG). MAG achieves the mutual attention mechanism between the center feature and surrounding feature with the center attention guidance module (CAG) and the surrounding attention guidance module (SAG). CAG aligns the center-view feature with the surrounding-view feature and generates the refined surrounding-view feature, while SAG aligns the refined surrounding-view feature with the original surrounding-view feature to implement bidirectional center-view, and surrounding view features alignment. Based on MAG, we build a Light Field Mutual Attention Guidance Network (LF-MAGNet) constructed by multiple MAGs in a cascade manner. Experiments are performed on commonly-used light field image super-resolution benchmarks. Our method achieves superior qualitative and quantitative results to other state-of-the-art methods, which demonstrate the effectiveness of our LF-MAGNet.INDEX TERMS Light-field image super-resolution, Visual attention mechanism, Feature alignment.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Proposed Methodsmentioning

confidence: 99%

Section: A Light Field Image Super-resolutionmentioning

confidence: 99%

See 2 more Smart Citations

Light Field Image Super-Resolution via Mutual Attention Guidance

Wang

2021

IEEE Access

View full text Add to dashboard Cite

show abstract

“…In detail, convolutional kernel with explicit offsets learned from the previous feature maps can adaptively change predefined receptive field in order to extract more target features. The specific deformable convolution is shown in Figure 2 , in which some standard convolution layers are first utilized to learn and regress the deformation displacements for each sampling point in the image, and then the learned displacements are added to original sampling positions of the 2D convolution to enable network extract relevant and rich features far from original fixed neighborhood [ 30 ]. Different from STN [ 28 ], deformable convolution adopts a local and dense, instead of global manner to warp feature maps.…”

Section: Introductionmentioning

confidence: 99%

A Semiautomated Deep Learning Approach for Pancreas Segmentation

Huang

Yuan

et al. 2021

Journal of Healthcare Engineering

View full text Add to dashboard Cite

Accurate pancreas segmentation from 3D CT volumes is important for pancreas diseases therapy. It is challenging to accurately delineate the pancreas due to the poor intensity contrast and intrinsic large variations in volume, shape, and location. In this paper, we propose a semiautomated deformable U-Net, i.e., DUNet for the pancreas segmentation. The key innovation of our proposed method is a deformable convolution module, which adaptively adds learned offsets to each sampling position of 2D convolutional kernel to enhance feature representation. Combining deformable convolution module with U-Net enables our DUNet to flexibly capture pancreatic features and improve the geometric modeling capability of U-Net. Moreover, a nonlinear Dice-based loss function is designed to tackle the class-imbalanced problem in the pancreas segmentation. Experimental results show that our proposed method outperforms all comparison methods on the same NIH dataset.

show abstract