HST: Hierarchical Swin Transformer for Compressed Image Super-Resolution

Li, Bingchen; Li, Xin; Lu, Yiting; Liu, Sen; Feng, Ruoyu; Chen, Zhibo

doi:10.1007/978-3-031-25063-7_41

Cited by 23 publications

(4 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Indeed, recent advancements in the field of super‐resolution have witnessed the emergence of various techniques harnessing transformers and diffusion models. Transformer‐based super‐resolution methods, such as those leveraging vision transformer (ViT) (Alimanov et al, 2023), Swin (Li, Li, et al, 2022), and efficient transformers (Lu, Li, et al, 2022), are among the notable approaches. Additionally, diffusion‐based super‐resolution techniques, like SRDiff (Li, Yang, et al, 2022) and implicit diffusion models (Gao et al, 2023), have also been developed.…”

Section: Literature Surveymentioning

confidence: 99%

A novel generative adversarial network‐based super‐resolution approach for face recognition

Chougule,

Kolte,

Chamola

et al. 2024

Expert Systems

View full text Add to dashboard Cite

Face recognition is an essential feature required for a range of computer vision applications such as security, attendance systems, emotion detection, airport check‐in, and many others. The super‐resolution of subject images is an important and challenging element in numerous scenarios. At times the images are low resolution and need to be processed through super‐resolution techniques to gain more accurate results. For the problem of image super‐resolution, deep learning‐based face recognition systems have been explored in recent years; however, low‐resolution face recognition remains an arduous task. Generative adversarial network (GAN) based models are a promising approach to address this challenge. However, conventional GAN‐based models may generate images that differ significantly from an original high‐resolution image in the test set to the point that the identity of the target face may be changed. To address this shortcoming, we propose a novel U‐Net style generator architecture, where skip‐connections between the encoder and decoder layer can help in preserving the facial characteristics of the input image in the generated image, thus curbing the generator's ability to generate an entirely new image and training it to generate an image more similar in characteristics to the original image. In addition to statistical metrics like structural similarity index measure and Fréchet inception distance, we compute the pixel‐wise distance between the original and model‐generated images to ascertain that our model generates as close to the original images as possible. While we train the model for 4× super‐resolution (64 × 64 images to 256 × 256), our architecture can also be trained for an arbitrary resizing scale. Finally, the number of faces detected over high‐resolution images generated by our model is shown to be higher than state‐of‐the‐art high‐resolution image creation models for face recognition tasks.

show abstract

Section: Literature Surveymentioning

confidence: 99%

A novel generative adversarial network‐based super‐resolution approach for face recognition

Chougule,

Kolte,

Chamola

et al. 2024

Expert Systems

View full text Add to dashboard Cite

show abstract

“…In [21], in order to recover the low-resolution compressed image, the authors have presented the Hierarchical SWIN Transformer (HST) network, which simultaneously captures the hierarchical feature representations and improves eachscale representation using SWIN transformer. In [22], the authors proposed a cross-modality fusion model, SWINNet, with the purpose of RGB-D and RGB-T salient object detection.…”

Section: Literature Surveymentioning

confidence: 99%

“…They developed SAIEC framework to improve segmentation accuracy. Overall, in image processing, SWIN Transformers demonstrate remarkable efficacy in tasks such as image restoration [13] and resolution scaling [14] [21]. Their versatility extends to the field of medical science, where they contribute to faster MRI processing [15], as well as enhancing medical image segmentation and analysis through integration into frameworks like U-Net [16][20] [23].…”

Section: Literature Surveymentioning

confidence: 99%

A Robust Approach for Deepfake Detection Using SWIN Transformer

Mishra,

Mohapatra,

Gourisaria

2024

Preprint

View full text Add to dashboard Cite

The widespread use of deepfake technology in recent years has made it extremely difficult to differentiate between real and fake images, usually AI-generated images. Effective detection techniques are desperately needed because one can generate fake images and spread them with ease. In response to the situation, this research paper explores the effectiveness of employing SWIN Transformer, a cutting-edge transformer-based architecture, for deep fake image detection. The foundation of the suggested detection framework is an architecture made up of bottleneck, encoder, and decoder parts which is a type of SWIN transformer. It uses various self-attention mechanisms and advanced features to analyse the images closely whether it is a real image or a deepfake one. It relies on the concept of shifted windows during the processing of the images and is considered more effective than the traditional CNN methods. Our test results show how well the SWIN Transformer-based method performs in precisely recognizing deep fake images. The accuracy is found to be 97.91% for CelebDF dataset and 95.715% for FF++ dataset. The AUC for the newly modelled SWIN transformer is 0.99 and 0.9625 for CelebDF and FF++ datasets respectively. The Log Loss has been calculated to be 0.034 for CelebDF dataset and 0.1573 for FF++ dataset. The proposed methodology not only enhances the accuracy of detecting manipulated images but also offers potential for scalable and efficient deployment in real-world scenarios where the proliferation of deepfakes presents significant challenges to maintaining trust and authenticity in visual media.

show abstract

“…Methods that use one input low-resolution (LR) image and produce one output high-resolution (HR) image are referred to as single-image super-resolution (SISR), while methods where multiple frames are fused to produce one HR image are referred to as multi-frame SR (MFSR). Several significant works documented in the literature [ 2 , 3 , 4 , 5 , 6 , 7 , 8 ] address SR for high-dimensional inputs, such as videos and 3D scans. The generation of HR images offers enriched details of locations and inherent objects, proving crucial in various applications, including high-definition TV sets, larger computer screens, and portable devices, like cameras, laptops, and mobile phones.…”

Section: Introductionmentioning

confidence: 99%

Exponential Fusion of Interpolated Frames Network (EFIF-Net): Advancing Multi-Frame Image Super-Resolution with Convolutional Neural Networks

Elwarfalli,

Flaute,

Hardie

2024

Sensors

View full text Add to dashboard Cite

Convolutional neural networks (CNNs) have become instrumental in advancing multi-frame image super-resolution (SR), a technique that merges multiple low-resolution images of the same scene into a high-resolution image. In this paper, a novel deep learning multi-frame SR algorithm is introduced. The proposed CNN model, named Exponential Fusion of Interpolated Frames Network (EFIF-Net), seamlessly integrates fusion and restoration within an end-to-end network. Key features of the new EFIF-Net include a custom exponentially weighted fusion (EWF) layer for image fusion and a modification of the Residual Channel Attention Network for restoration to deblur the fused image. Input frames are registered with subpixel accuracy using an affine motion model to capture the camera platform motion. The frames are externally upsampled using single-image interpolation. The interpolated frames are then fused with the custom EWF layer, employing subpixel registration information to give more weight to pixels with less interpolation error. Realistic image acquisition conditions are simulated to generate training and testing datasets with corresponding ground truths. The observation model captures optical degradation from diffraction and detector integration from the sensor. The experimental results demonstrate the efficacy of EFIF-Net using both simulated and real camera data. The real camera results use authentic, unaltered camera data without artificial downsampling or degradation.

show abstract

HST: Hierarchical Swin Transformer for Compressed Image Super-Resolution

Cited by 23 publications

References 50 publications

A novel generative adversarial network‐based super‐resolution approach for face recognition

A novel generative adversarial network‐based super‐resolution approach for face recognition

A Robust Approach for Deepfake Detection Using SWIN Transformer

Exponential Fusion of Interpolated Frames Network (EFIF-Net): Advancing Multi-Frame Image Super-Resolution with Convolutional Neural Networks

Contact Info

Product

Resources

About