Lightweight Modules for Efficient Deep Learning Based Image Restoration

Lahiri, Avisek; Bairagya, Sourav; Bera, Sutanu; Haldar, Siddhant; Biswas, Prabir Kumar

doi:10.1109/tcsvt.2020.3007723

Cited by 43 publications

(4 citation statements)

References 73 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When the size of the model reaches a certain level, it is difficult to obtain significant performance improvement by simply increasing the size of the model. Grouped convolution and depthwise convolution have been applied to many lightweight models (Zhou et al 2020, Lahiri et al 2021, and we also demonstrated their excellent ability in ECG denoising. ECG characteristic waveforms contain rich medical and physiological information, and it is important to retain effective characteristic waveforms while removing noise.…”

Section: Discussionmentioning

confidence: 75%

A lightweight U-net for ECG denoising using knowledge distillation

et al. 2022

View full text Add to dashboard Cite

Objective: Electrocardiogram (ECG) signals are easily polluted by various noises, which are likely to have adverse effects on subsequent interpretations. Research on model lightweighting can promote the practical application of deep learning-based ECG denoising methods in real-time processing. Approach: Firstly, grouped convolution and conventional convolution are combined to replace the continuous conventional convolution in the model, the depthwise convolution with stride is used to compress the feature map in the encoder modules. Secondly, additional identity connections and local maximum and minimum enhancement module are designed, which can retain the detailed information and characteristic waveform in the ECG waveform while effectively denoising. Finally, we develop knowledge distillation in the experiments, which further improves the ECG denoising performance without increasing the model complexity. The ground-truth ECG is from The China Physiological Signal Challenge (CPSC) 2018, and the noise signal is from the MIT-BIH Noise Stress Test Database (NSTDB). We evaluate denoising performance by the signal-to-noise ratio (SNR), the root mean square error (RMSE) and the pearson correlation coefficient (P). Using floating point of operations (FLOPs) and Parameters to calculate computational complexity. Main Results: Different data generation processes are used to conduct experiments: Group1, Group2 and Group3. The results show that the proposed model (ULde-net) can improve SNRs by 10.30 dB, 12.16 dB and 12.61 dB; reduce RMSEs by 9.88×10-2, 20.63×10-2 and 15.25×10-2; increase Ps by 14.77×10-2, 27.74×10-2 and 21.32×10-2. Moreover, the denoising performance after knowledge distillation is further improved. The ULde-net has parameters of 6.9 K and FLOPs of 6.6 M, which are much smaller than the compared models. Significance: We designed a lightweight model, but also retain the adequate ECG denoising performance. We believe that this method can be successfully applied to practical applications under time or memory limits.

show abstract

Section: Discussionmentioning

confidence: 75%

A lightweight U-net for ECG denoising using knowledge distillation

et al. 2022

View full text Add to dashboard Cite

show abstract

“…However, most of the methods are limited in real-world applications by their huge computational cost. To handle this issue, some lightweight and efficient SISR methods have been presented [17], [25]- [31]. For example, IDN [32] used an information distillation network to selectively fuse features, and then IMDN [33] improved it to build a lighter model.…”

Section: A Lightweight Sisr Methodsmentioning

confidence: 99%

Cross-receptive Focused Inference Network for Lightweight Image Super-Resolution

Li¹,

Li²,

Gao³

et al. 2022

Preprint

View full text Add to dashboard Cite

With the development of deep learning, single image super-resolution (SISR) has achieved significant breakthroughs. Recently, methods to enhance the performance of SISR networks based on global feature interactions have been proposed. However, the capabilities of neurons that need to adjust their function in response to the context dynamically are neglected. To address this issue, we propose a lightweight Cross-receptive Focused Inference Network (CFIN), a hybrid network composed of a Convolutional Neural Network (CNN) and a Transformer. Specifically, a novel Cross-receptive Field Guide Transformer (CFGT) is designed to adaptively modify the network weights by using modulated convolution kernels combined with local representative semantic information. In addition, a CNN-based Cross-scale Information Aggregation Module (CIAM) is proposed to make the model better focused on potentially practical information and improve the efficiency of the Transformer stage. Extensive experiments show that our proposed CFIN is a lightweight and efficient SISR model, which can achieve a good balance between computational cost and model performance.

show abstract

“…It is noted that the differences in terms of image quality and video specification between the clean virtual contents and real displayed images/videos can appear in VR devices [66], [67]. These gaps can be filled effectively with AI-empowered image restoration methods, such as blur estimation, hazy removal, color correction, and texture reconstruction, but the computational complexity should satisfy the real-time video processing speed (usually measured by frames per second -FPS metric) [68] to guarantee high-class user experience in the metaverse. Image enhancement has been widely considered for XR with some common tasks, such as contrast increment and superresolution construction.…”

Section: B Machine Visionmentioning

confidence: 99%

Artificial Intelligence for the Metaverse: A Survey

Huynh‐The¹,

Pham²,

Pham³

et al. 2022

Preprint

View full text Add to dashboard Cite

Along with the massive growth of the Internet from the 1990s until now, various innovative technologies have been created to bring users breathtaking experiences with more virtual interactions in cyberspace. Many virtual environments with thousands of services and applications, from social networks to virtual gaming worlds, have been developed with immersive experience and digital transformation, but most are incoherent instead of being integrated into a platform. In this context, metaverse, a term formed by combining meta and universe, has been introduced as a shared virtual world that is fueled by many emerging technologies, such as fifth-generation networks and beyond, virtual reality, and artificial intelligence (AI). Among such technologies, AI has shown the great importance of processing big data to enhance immersive experience and enable humanlike intelligence of virtual agents. In this survey, we make a beneficial effort to explore the role of AI in the foundation and development of the metaverse. We first deliver a preliminary of AI, including machine learning algorithms and deep learning architectures, and its role in the metaverse. We then convey a comprehensive investigation of AI-based methods concerning six technical aspects that have potentials for the metaverse: natural language processing, machine vision, blockchain, networking, digital twin, and neural interface, and being potential for the metaverse. Subsequently, several AI-aided applications, such as healthcare, manufacturing, smart cities, and gaming, are studied to be deployed in the virtual worlds. Finally, we conclude the key contribution of this survey and open some future research directions in AI for the metaverse.

show abstract

Lightweight Modules for Efficient Deep Learning Based Image Restoration

Cited by 43 publications

References 73 publications

A lightweight U-net for ECG denoising using knowledge distillation

A lightweight U-net for ECG denoising using knowledge distillation

Cross-receptive Focused Inference Network for Lightweight Image Super-Resolution

Artificial Intelligence for the Metaverse: A Survey

Contact Info

Product

Resources

About