Local-Selective Feature Distillation for Single Image Super-Resolution

Park, SeongUk; Kwak, Nojun

doi:10.48550/arxiv.2111.10988

Cited by 3 publications

(16 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, in generative tasks that produce an image (such as super-resolution and demosaicing), directly applying distillation techniques is challenging since they do not have a notion of soft labels. Consequently, many super-resolution networks distill teacher networks' features [55], [56], [57] using an additional tool like a regressor to address dimension mismatches between the teacher and student. Gao et al [55] transfer the first-order statistical map (e.g., average, maximum, or minimum value) of intermediate features, while FAKD [56] proposed spatial affinity of features-based distillation to utilize rich high-dimensional statistical information.…”

Section: Knowledge Distillationmentioning

confidence: 99%

“…Gao et al [55] transfer the first-order statistical map (e.g., average, maximum, or minimum value) of intermediate features, while FAKD [56] proposed spatial affinity of features-based distillation to utilize rich high-dimensional statistical information. LSFD [57] introduced a deeper regressor comprising 3 × 3 convolution layers to achieve a larger receptive field and an attention method based on the difference (between teacher and student) that selectively focuses on vulnerable pixel locations. PISR [58] added an encoder that leverages privileged information from ground truth and transfers knowledge through feature distillation.…”

Section: Knowledge Distillationmentioning

confidence: 99%

“…For feature distillation, we calculate the L 2 difference between intermediate features. However, the teacher and student have different spatial dimensions of features, which is why previous studies [55], [56], [57] introduced additional regressors or extracted statistical information from features. We also add 1 × 1 regressors R and compare the intermediate features of level 1.…”

Section: S and I (1)mentioning

confidence: 99%

See 2 more Smart Citations

PyNET-Q×Q: An Efficient PyNET Variant for Q×Q Bayer Pattern Demosaicing in CMOS Image Sensors

Cho

Lee

et al. 2023

IEEE Access

View full text Add to dashboard Cite

Deep learning-based image signal processor (ISP) models for mobile cameras can generate high-quality images that rival those of professional DSLR cameras. However, their computational demands often make them unsuitable for mobile settings. Additionally, modern mobile cameras employ non-Bayer color filter arrays (CFA) such as Quad Bayer, Nona Bayer, and Q×Q Bayer to enhance image quality, yet most existing deep learning-based ISP (or demosaicing) models focus primarily on standard Bayer CFAs. In this study, we present PyNET-Q×Q, a lightweight demosaicing model specifically designed for Q×Q Bayer CFA patterns, which is derived from the original PyNET. We also propose a knowledge distillation method called progressive distillation to train the reduced network more effectively. Consequently, PyNET-Q×Q contains less than 2.5% of the parameters of the original PyNET while preserving its performance. Experiments using Q×Q images captured by a prototype Q×Q camera sensor show that PyNET-Q×Q outperforms existing conventional algorithms in terms of texture and edge reconstruction, despite its significantly reduced parameter count. Code and partial datasets can be found at https://github.com/Minhyeok01/PyNET-QxQ.INDEX TERMS Bayer filter, color filter array (CFA), demosaicing, image signal processor (ISP), knowledge distillation, non-Bayer CFA, Q×Q Bayer CFA.

show abstract

Section: Knowledge Distillationmentioning

confidence: 99%

Section: Knowledge Distillationmentioning

confidence: 99%

Section: S and I (1)mentioning

confidence: 99%

See 1 more Smart Citation

PyNET-Q×Q: An Efficient PyNET Variant for Q×Q Bayer Pattern Demosaicing in CMOS Image Sensors

Cho

Lee

et al. 2023

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Knowledge distillation transfers rich knowledge from teacher models to lightweight student models in the label or feature domains, mainly applied in model compression and acceleration [15,42]. Among various approaches, featurebased knowledge distillation is extensively applied in image restoration tasks such as image super-resolution [14,16,27,38]. It minimizes the distance between feature representations to learn richer information from the teacher model compared to softened labels [49].…”

Section: Knowledge Distillationmentioning

confidence: 99%

Transfer Learning in Motor Imagery Brain Computer Interface: A Review

2022

J. Shanghai Jiaotong Univ. (Sci.)

View full text Add to dashboard Cite

Sparse-view computed tomography (CT)-using a small number of projections for tomographic reconstructionenables much lower radiation dose to patients and accelerated data acquisition. The reconstructed images, however, suffer from strong artifacts, greatly limiting their diagnostic value. Current trends for sparse-view CT turn to the raw data for better information recovery. The resultant dual-domain methods, nonetheless, suffer from secondary artifacts, especially in ultra-sparse view scenarios, and their generalization to other scanners/protocols is greatly limited. A crucial question arises: have the image post-processing methods reached the limit? Our answer is not yet. In this paper, we stick to image post-processing methods due to great flexibility and propose global representation(GloRe) distillation framework for sparse-view CT, termed GloReDi. First, we propose to learn GloRe with Fourier convolution, so each element in GloRe has an image-wide receptive field. Second, unlike methods that only use the full-view images for supervision, we propose to distill GloRe from intermediate-view reconstructed images that are readily available but not explored in previous literature. The success of GloRe distillation is attributed to two key components: representation directional distillation to align the GloRe directions, and band-pass-specific contrastive distillation to gain clinically important details. Extensive experiments demonstrate the superiority of the proposed GloReDi over the state-of-the-art methods, including dual-domain ones. The source code is available at https: //github.com/longzilicart/GloReDi.

show abstract

“…The practical applicability of complex SISR models is limited in resource-constrained devices, such as mobile or IoT devices; thus, efficient and lightweight SISR models are required. To satisfy this demand, lightweight SISR models with a better trade-off between efficiency and performance quality have been proposed [16], [17], [19]- [25]. Among the aforementioned methods, knowledge distillation (KD) [26]based approaches exhibit the following distinctive advantages: 1) KD promotes the inheritance of the knowledge of large teacher networks and improves performance without modifying the existing network structure at industrial sites, and 2) KD can be combined with pruning and network design methods by including additional loss terms to achieve greater performance improvement [27], [28].…”

Section: Introductionmentioning

confidence: 99%

Feature-Domain Adaptive Contrastive Distillation for Efficient Single Image Super-Resolution

Moon,

Kim,

Jeong

et al. 2023

IEEE Access

View full text Add to dashboard Cite

Convolutional neural network-based single image super-resolution (SISR) involves numerous parameters and high computational expenses to ensure improved performance, limiting its applicability in resource-constrained devices such as mobile phones. Knowledge distillation (KD), which transfers useful knowledge from a teacher network to a student network, has been investigated as a method to make networks more efficient in terms of performance. To this end, feature distillation (FD) has been utilized in KD to minimize the Euclidean distance-based loss of feature maps between teacher and student networks. However, this technique does not adequately consider the effective and meaningful delivery of knowledge from the teacher to the student network to improve the latter's performance under given network capacity constraints. In this study, we propose a feature-domain adaptive contrastive distillation (FACD) method to train lightweight student SISR networks efficiently. We highlight the limitations of existing FD methods in terms of Euclidean distance-based loss, and propose a feature-domain contrastive loss, which causes student networks to learn richer information from the teacher's representation in the feature domain. We also implement adaptive distillation that performs distillation selectively depending on the conditions of the training patches. Experimental results showed that the proposed FACD scheme improves student enhanced deep residual networks and residual channel attention networks not only in terms of the peak signal-to-noise ratio on all benchmark datasets and scales but also in terms of subjective image quality, compared to the conventional FD approaches. Code will be release at https://github.com/hcmoon0613/FACD.

show abstract

Local-Selective Feature Distillation for Single Image Super-Resolution

Cited by 3 publications

References 43 publications

PyNET-Q×Q: An Efficient PyNET Variant for Q×Q Bayer Pattern Demosaicing in CMOS Image Sensors

PyNET-Q×Q: An Efficient PyNET Variant for Q×Q Bayer Pattern Demosaicing in CMOS Image Sensors

Transfer Learning in Motor Imagery Brain Computer Interface: A Review

Feature-Domain Adaptive Contrastive Distillation for Efficient Single Image Super-Resolution

Contact Info

Product

Resources

About