Video Super-Resolution Method Using Deformable Convolution-Based Alignment Network

Lee, Yooho; Cho, Sukhee; Jun, Dongsan

doi:10.3390/s22218476

Cited by 2 publications

(2 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…63 Moreover, in view of the drawback exhibited by a fixed grid kernel, the receptive field cannot adaptively change when handling different-structured information in the representation space, as shown in the middle of Figure 3. We introduce deformable convolution 64 to replace part of the normal convolution operations to form the designed DEM. The right side of Figure 3 illustrates the advantages of the deformable convolution operation.…”

Section: Demmentioning

confidence: 99%

Total‐body low‐dose CT image denoising using a prior knowledge transfer technique with a contrastive regularization mechanism

Duan

Cheng

et al. 2023

Medical Physics

View full text Add to dashboard Cite

Reducing the radiation exposure experienced by patients in totalbody computed tomography (CT) imaging has attracted extensive attention in the medical imaging community. A low radiation dose may result in increased noise and artifacts that greatly affect the subsequent clinical diagnosis.To obtain high-quality total-body low-dose CT (LDCT) images, previous deep learningbased research works developed various network architectures. However, most of these methods only employ normal-dose CT (NDCT) images as ground truths to guide the training process of the constructed denoising network. As a result of this simple restriction, the reconstructed images tend to lose favorable image details and easily generate oversmoothed textures. This study explores how to better utilize the information contained in the feature spaces of NDCT images to guide the LDCT image reconstruction process and achieve high-quality results. Methods: We propose a novel intratask knowledge transfer (KT) method that leverages the knowledge distilled from NDCT images as an auxiliary component of the LDCT image reconstruction process. Our proposed architecture is named the teacher-student consistency network (TSC-Net), which consists of teacher and student networks with identical architectures. By employing the designed KT loss, the student network is encouraged to emulate the teacher network in the representation space and gain robust prior content. In addition, to further exploit the information contained in CT scans, a contrastive regularization mechanism (CRM) built upon contrastive learning is introduced. The CRM aims to minimize and maximize the L2 distances from the predicted CT images to the NDCT samples and to the LDCT samples in the latent space, respectively. Moreover, based on attention and the deformable convolution approach, we design a dynamic enhancement module (DEM) to improve the network capability to transform input information flows. Results: By conducting ablation studies, we prove the effectiveness of the proposed KT loss, CRM, and DEM. Extensive experimental results demonstrate that the TSC-Net outperforms the state-of -the-art methods in both quantitative and qualitative evaluations. Additionally, the excellent results obtained for clinical readings also prove that our proposed method can reconstruct high-quality CT images for clinical applications. Conclusions: Based on the experimental results and clinical readings,the TSC-Net has better performance than other approaches. In our future work, we may explore the reconstruction of LDCT images by fusing the positron emission

show abstract

Section: Demmentioning

confidence: 99%

Total‐body low‐dose CT image denoising using a prior knowledge transfer technique with a contrastive regularization mechanism

Duan

Cheng

et al. 2023

Medical Physics

View full text Add to dashboard Cite

show abstract

“…Liu et al [44] applied spatial convolution packing to jointly exploit spatial-temporal features. For better fusing information from neighboring frames, Lee et al [45] utilized both attention-based alignment and dilation-based alignment. Lian et al [18] proposed SWRN to achieve real-time inference while producing superior performance.…”

Section: Video Super-resolutionmentioning

confidence: 99%

Adapting Single-Image Super-Resolution Models to Video Super-Resolution: A Plug-and-Play Approach

Wang

Liu

et al. 2023

Sensors

View full text Add to dashboard Cite

The quality of videos varies due to the different capabilities of sensors. Video super-resolution (VSR) is a technology that improves the quality of captured video. However, the development of a VSR model is very costly. In this paper, we present a novel approach for adapting single-image super-resolution (SISR) models to the VSR task. To achieve this, we first summarize a common architecture of SISR models and perform a formal analysis of adaptation. Then, we propose an adaptation method that incorporates a plug-and-play temporal feature extraction module into existing SISR models. The proposed temporal feature extraction module consists of three submodules: offset estimation, spatial aggregation, and temporal aggregation. In the spatial aggregation submodule, the features obtained from the SISR model are aligned to the center frame based on the offset estimation results. The aligned features are fused in the temporal aggregation submodule. Finally, the fused temporal feature is fed to the SISR model for reconstruction. To evaluate the effectiveness of our method, we adapt five representative SISR models and evaluate these models on two popular benchmarks. The experiment results show the proposed method is effective on different SISR models. In particular, on the Vid4 benchmark, the VSR-adapted models achieve at least 1.26 dB and 0.067 improvement over the original SISR models in terms of PSNR and SSIM metrics, respectively. Additionally, these VSR-adapted models achieve better performance than the state-of-the-art VSR models.

show abstract

Video Super-Resolution Method Using Deformable Convolution-Based Alignment Network

Cited by 2 publications

References 53 publications

Total‐body low‐dose CT image denoising using a prior knowledge transfer technique with a contrastive regularization mechanism

Total‐body low‐dose CT image denoising using a prior knowledge transfer technique with a contrastive regularization mechanism

Adapting Single-Image Super-Resolution Models to Video Super-Resolution: A Plug-and-Play Approach

Contact Info

Product

Resources

About