Dynamic Depth-Aware Network for Endoscopy Super-Resolution

Chen, Wenting; Liu, Yifan; Hu, Jiancong; Yuan, Yixuan

doi:10.1109/jbhi.2022.3188878

Cited by 14 publications

(3 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…DL-SR methods that effectively estimate SR images, resembling the original HR images, is a robust strategy to increase the field of visualization during EC screening while maintaining accurate esophageal neoplasia detection by clinicians. More advanced deep-learning-based approaches for endoscopy super-resolution, such as those employing deformable transformer, contrastive adversarial learning, and zero-shot learning 11 , 12 , 29 – 32 should also be investigated for the end-expandable probe application in future studies.…”

Section: Discussionmentioning

confidence: 99%

Deep-learning-based image super-resolution of an end-expandable optical fiber probe for application in esophageal cancer diagnostics

Zhang,

Tan,

Nabil

et al. 2024

J. Biomed. Opt.

View full text Add to dashboard Cite

Endoscopic screening for esophageal cancer (EC) may enable early cancer diagnosis and treatment. While optical microendoscopic technology has shown promise in improving specificity, the limited field of view (<1 mm) significantly reduces the ability to survey large areas efficiently in EC screening.Aim: To improve the efficiency of endoscopic screening, we propose a novel concept of end-expandable endoscopic optical fiber probe for larger field of visualization and for the first time evaluate a deep-learning-based image super-resolution (DL-SR) method to overcome the issue of limited sampling capability.Approach: To demonstrate feasibility of the end-expandable optical fiber probe, DL-SR was applied on simulated low-resolution microendoscopic images to generate super-resolved (SR) ones. Varying the degradation model of image data acquisition, we identified the optimal parameters for optical fiber probe prototyping. The proposed screening method was validated with a human pathology reading study.Results: For various degradation parameters considered, the DL-SR method demonstrated different levels of improvement of traditional measures of image quality. The endoscopists' interpretations of the SR images were comparable to those performed on the high-resolution ones.Conclusions: This work suggests avenues for development of DL-SR-enabled sparse image reconstruction to improve high-yield EC screening and similar clinical applications.

show abstract

Section: Discussionmentioning

confidence: 99%

Deep-learning-based image super-resolution of an end-expandable optical fiber probe for application in esophageal cancer diagnostics

Zhang,

Tan,

Nabil

et al. 2024

J. Biomed. Opt.

View full text Add to dashboard Cite

show abstract

“…With the significant progress of deep learning (DL) in computer vision tasks, various medical image SR methods have begun to adopt deep neural network for super-resolution, such as the convolution-based [2], [16], [22], [24], [33]- [36] and GAN-based [19], [33], [37]- [39] methods. For instance, EndoL2H [37], a GAN (Generative Adversarial Nets) [40] based framework, introduced an attention U-Net to map the LR to HR image and a discriminator to distinguish the SR from HR image.…”

Section: Related Workmentioning

confidence: 99%

Hierarchical Generative Adversarial Networks for Single Image Super-Resolution

Chen

Liu

2021

2021 IEEE Winter Conference on Applications of Computer Vision (WACV)

View full text Add to dashboard Cite

Pathology image are essential for accurately interpreting lesion cells in cytopathology screening, but acquiring high-resolution digital slides requires specialized equipment and long scanning times. Though superresolution (SR) techniques can alleviate this problem, existing deep learning models recover pathology image in a black-box manner, which can lead to untruthful biological details and misdiagnosis. Additionally, current methods allocate the same computational resources to recover each pixel of pathology image, leading to the sub-optimal recovery issue due to the large variation of pathology image. In this paper, we propose the first hierarchical reinforcement learning framework named Spatial-Temporal hierARchical Reinforcement Learning (STAR-RL), mainly for addressing the aforementioned issues in pathology image super-resolution problem. We reformulate the SR problem as a Markov decision process of interpretable operations and adopt the hierarchical recovery mechanism in patch level, to avoid sub-optimal recovery. Specifically, the higher-level spatial manager is proposed to pick out the most corrupted patch for the lower-level patch worker. Moreover, the higher-level temporal manager is advanced to evaluate the selected patch and determine whether the optimization should be stopped earlier, thereby avoiding the over-processed problem. Under the guidance of spatialtemporal managers, the lower-level patch worker processes the selected patch with pixel-wise interpretable actions at each time step. Experimental results on medical images degraded by different kernels show the effectiveness of STAR-RL. Furthermore, STAR-RL validates the promotion in tumor diagnosis with a large margin and shows generalizability under various degradations. The source code is available at https://github.com/CUHK-AIM-Group/

show abstract

“…However, DNNs are vulnerable to adversarial samples [47], which are elaborately designed by adding humanimperceptible noise to the clean image to mislead DNNs into wrong predictions. The existence of adversarial samples causes negative effects on security-sensitive DNNbased applications, such as self-driving [27] and medical diagnosis [6,7]. Therefore, it is necessary to understand the DNNs [15,32,33,40] and enhance attack algorithms to better identify the DNN model's vulnerability, which is the first step to improve their robustness against adversarial samples [23,24,42].…”

Section: Introductionmentioning

confidence: 99%

Improving the Transferability of Adversarial Samples by Path-Augmented Method

Zhang¹,

Huang²,

Wang³

et al. 2023

Preprint

View full text Add to dashboard Cite

Deep neural networks have achieved unprecedented success on diverse vision tasks. However, they are vulnerable to adversarial noise that is imperceptible to humans. This phenomenon negatively affects their deployment in real-world scenarios, especially security-related ones. To evaluate the robustness of a target model in practice, transfer-based attacks craft adversarial samples with a local model and have attracted increasing attention from researchers due to their high efficiency. The state-of-the-art transfer-based attacks are generally based on data augmentation, which typically augments multiple training images from a linear path when learning adversarial samples. However, such methods selected the image augmentation path heuristically and may augment images that are semantics-inconsistent with the target images, which harms the transferability of the generated adversarial samples. To overcome the pitfall, we propose the Path-Augmented Method (PAM). Specifically, PAM first constructs a candidate augmentation path pool. It then settles the employed augmentation paths during adversarial sample generation with greedy search. Furthermore, to avoid augmenting semantics-inconsistent images, we train a Semantics Predictor (SP) to constrain the length of the augmentation path. Extensive experiments confirm that PAM can achieve an improvement of over 4.8% on average compared with the state-of-the-art baselines in terms of the attack success rates.

show abstract

Dynamic Depth-Aware Network for Endoscopy Super-Resolution

Cited by 14 publications

References 42 publications

Deep-learning-based image super-resolution of an end-expandable optical fiber probe for application in esophageal cancer diagnostics

Deep-learning-based image super-resolution of an end-expandable optical fiber probe for application in esophageal cancer diagnostics

Hierarchical Generative Adversarial Networks for Single Image Super-Resolution

Improving the Transferability of Adversarial Samples by Path-Augmented Method

Contact Info

Product

Resources

About