TC-net: transformer combined with cnn for image denoising

Xue, Tao; Ma, Pengsen

doi:10.1007/s10489-022-03785-w

Cited by 13 publications

(3 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Xu et al in [103] proposed the CUR transformer for image denoising.The CUR transformer was deduced from the convolutional unbiased regional transformer. Similarly, in [104][105][106], the combined transformers and CNN for image denoising and achieved better performance. In [107], the authors proposed Hider, a transformer-based model for image denoising.…”

Section: Vits For Image Denoisingmentioning

confidence: 94%

A Comprehensive Survey of Transformers for Computer Vision

Jamil

Piran

Kwon

2023

Drones

View full text Add to dashboard Cite

As a special type of transformer, vision transformers (ViTs) can be used for various computer vision (CV) applications. Convolutional neural networks (CNNs) have several potential problems that can be resolved with ViTs. For image coding tasks such as compression, super-resolution, segmentation, and denoising, different variants of ViTs are used. In our survey, we determined the many CV applications to which ViTs are applicable. CV applications reviewed included image classification, object detection, image segmentation, image compression, image super-resolution, image denoising, anomaly detection, and drone imagery. We reviewed the state of the-art and compiled a list of available models and discussed the pros and cons of each model.

show abstract

Section: Vits For Image Denoisingmentioning

confidence: 94%

A Comprehensive Survey of Transformers for Computer Vision

Jamil

Piran

Kwon

2023

Drones

View full text Add to dashboard Cite

show abstract

“…With relatively low computational costs, TECDNet achieves state-of-the-art denosing performance on real images. Similarly, this paper [47] proposes a novel and effective network architecture based on the transformer TC-Net. For image denoising, the architecture consists of several transformer blocks and convolutions.…”

Section: Approaches Embedding Image Denoising In Deep Learningmentioning

confidence: 99%

Impact of Traditional and Embedded Image Denoising on CNN-Based Deep Learning

Kaur,

Karmakar,

Imran

2023

Applied Sciences

View full text Add to dashboard Cite

In digital image processing, filtering noise is an important step for reconstructing a high-quality image for further processing such as object segmentation, object detection, and object recognition. Various image-denoising approaches, including median, Gaussian, and bilateral filters, are available in the literature. Since convolutional neural networks (CNN) are able to directly learn complex patterns and features from data, they have become a popular choice for image-denoising tasks. As a result of their ability to learn and adapt to various denoising scenarios, CNNs are powerful tools for image denoising. Some deep learning techniques such as CNN incorporate denoising strategies directly into the CNN model layers. A primary limitation of these methods is their necessity to resize images to a consistent size. This resizing can result in a loss of vital image details, which might compromise CNN’s effectiveness. Because of this issue, we utilize a traditional denoising method as a preliminary step for noise reduction before applying CNN. To our knowledge, a comparative performance study of CNN using traditional and embedded denoising against a baseline approach (without denoising) is yet to be performed. To analyze the impact of denoising on the CNN performance, in this paper, firstly, we filter the noise from the images using traditional means of denoising method before their use in the CNN model. Secondly, we embed a denoising layer in the CNN model. To validate the performance of image denoising, we performed extensive experiments for both traffic sign and object recognition datasets. To decide whether denoising will be adopted and to decide on the type of filter to be used, we also present an approach exploiting the peak-signal-to-noise-ratio (PSNRs) distribution of images. Both CNN accuracy and PSNRs distribution are used to evaluate the effectiveness of the denoising approaches. As expected, the results vary with the type of filter, impact, and dataset used in both traditional and embedded denoising approaches. However, traditional denoising shows better accuracy, while embedded denoising shows lower computational time for most of the cases. Overall, this comparative study gives insights into whether denoising will be adopted in various CNN-based image analyses, including autonomous driving, animal detection, and facial recognition.

show abstract

“…Additionally, denoising has been extensively researched. Denoising approaches based on self-supervised learning [17,18] via the noise-removal process can effectively capture features and learn deep representations. They have many similarities with self-supervised pre-training strategies, thus making the integration of the denoising concept into pretraining feasible.…”

Section: Introductionmentioning

confidence: 99%

Deformer: Denoising Transformer for Improved Audio Music Genre Classification

Wang,

Li,

Sung

2023

Applied Sciences

View full text Add to dashboard Cite

Audio music genre classification is performed to categorize audio music into various genres. Traditional approaches based on convolutional recurrent neural networks do not consider long temporal information, and their sequential structures result in longer training times and convergence difficulties. To overcome these problems, a traditional transformer-based approach was introduced. However, this approach employs pre-training based on momentum contrast (MoCo), a technique that increases computational costs owing to its reliance on extracting many negative samples and its use of highly sensitive hyperparameters. Consequently, this complicates the training process and increases the risk of learning imbalances between positive and negative sample sets. In this paper, a method for audio music genre classification called Deformer is proposed. The Deformer learns deep representations of audio music data through a denoising process, eliminating the need for MoCo and additional hyperparameters, thus reducing computational costs. In the denoising process, it employs a prior decoder to reconstruct the audio patches, thereby enhancing the interpretability of the representations. By calculating the mean squared error loss between the reconstructed and real patches, Deformer can learn a more refined representation of the audio data. The performance of the proposed method was experimentally compared with that of two distinct baseline models: one based on S3T and one employing a residual neural network-bidirectional gated recurrent unit (ResNet-BiGRU). The Deformer achieved an 84.5% accuracy, surpassing both the ResNet-BiGRU-based (81%) and S3T-based (81.1%) models, highlighting its superior performance in audio classification.

show abstract

TC-net: transformer combined with cnn for image denoising

Cited by 13 publications

References 39 publications

A Comprehensive Survey of Transformers for Computer Vision

A Comprehensive Survey of Transformers for Computer Vision

Impact of Traditional and Embedded Image Denoising on CNN-Based Deep Learning

Deformer: Denoising Transformer for Improved Audio Music Genre Classification

Contact Info

Product

Resources

About