Adversarially Robust Prototypical Few-Shot Segmentation with Neural-ODEs

Pandey, Prashant; Vardhan, Aleti; Chasmai, Mustafa; Sur, Tanuj; Lall, Brejesh

doi:10.1007/978-3-031-16452-1_8

Cited by 9 publications

(5 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When considering the robustness of our HPAN to adversarial attacks [69], [70] in real-world applications, it is a growing concern within the community. To counter these threats, several adversarial defense strategies have been put forward, such as image restoration [71] and adversarial training [72], all aimed at bolstering the defense capabilities of the network. Therefore, exploring adversarial attacks and defenses for FSVOS in practical scenarios has emerged as a significant area of research, highlighting our future potential improvement points.…”

Section: A Limitation and Future Workmentioning

confidence: 99%

Holistic Prototype Attention Network for Few-Shot Video Object Segmentation

Tang

Chen

Jiang

et al. 2024

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

Few-shot video object segmentation (FSVOS) aims to segment dynamic objects of unseen classes by resorting to a small set of support images that contain pixel-level object annotations. Existing methods have demonstrated that the domain agent-based attention mechanism is effective in FSVOS by learning the correlation between support images and query frames. However, the agent frame contains redundant pixel information and background noise, resulting in inferior segmentation performance. Moreover, existing methods tend to ignore inter-frame correlations in query videos. To alleviate the above dilemma, we propose a holistic prototype attention network (HPAN) for advancing FSVOS. Specifically, HPAN introduces a prototype graph attention module (PGAM) and a bidirectional prototype attention module (BPAM), transferring informative knowledge from seen to unseen classes. PGAM generates local prototypes from all foreground features and then utilizes their internal correlations to enhance the representation of the holistic prototypes. BPAM exploits the holistic information from support images and video frames by fusing co-attention and self-attention to achieve support-query semantic consistency and inner-frame temporal consistency. Extensive experiments on YouTube-FSVOS have been provided to demonstrate the effectiveness and superiority of our proposed HPAN method. Our source code and models are available anonymously at https: //github.com/NUST-Machine-Intelligence-Laboratory/HPAN.

show abstract

Section: A Limitation and Future Workmentioning

confidence: 99%

Holistic Prototype Attention Network for Few-Shot Video Object Segmentation

Tang

Chen

Jiang

et al. 2024

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

show abstract

“…Few-shot segmentation (FSS) [41] provides a potential solution that accomplishes segmentation for novel classes in a query image trained on a few annotated support images. However, the robustness of the FSS against the adversarial attack is unclear, which motivates Pandey et al [42] to combine FSS with NODE to build a data-efficient segmentation model that is both accurate and safe. The method proposed by Pandey et al [42] is named regularized prototypical neural ordinary differential equation (R-PNODE), designed to leverage the intrinsic properties of NODE for FSS of organs in computed tomography (CT).…”

Section: Applications In Medical Image Segmentationmentioning

confidence: 99%

“…However, the robustness of the FSS against the adversarial attack is unclear, which motivates Pandey et al [42] to combine FSS with NODE to build a data-efficient segmentation model that is both accurate and safe. The method proposed by Pandey et al [42] is named regularized prototypical neural ordinary differential equation (R-PNODE), designed to leverage the intrinsic properties of NODE for FSS of organs in computed tomography (CT). For the paired query and support images denoted as (x q , x s ), the authors first obtain their corrupted version by applying Gaussian noise that is formulated as (x G q , x G s ).…”

Section: Applications In Medical Image Segmentationmentioning

confidence: 99%

On the Applications of Neural Ordinary Differential Equations in Medical Image Analysis

Niu,

Zhou,

Yan

et al. 2024

Preprint

View full text Add to dashboard Cite

Medical image analysis tasks are characterized by high-noise, volumetric, and multi-modality, posing challenges for the model that attempts to learn robust features from the input images. Over the last decade, deep neural networks (DNNs) have achieved enormous success in medical image analysis tasks, which can be attributed to their powerful feature representation capability. Despite the promising results reported in numerous literature, DNNs are also criticized for several pivotal limits, with one of the limitations is lack of safety. Safety plays an important role in the applications of DNNs during clinical practice, helping the model defend against potential attacks and preventing the model from silent failure prediction. The recently proposed Neural Ordinary Differential Equation (NODE), a continuous model bridging the gap between DNNs and ODE, provides a significant advantage in ensuring the model's safety. Among the variants of NODE, the neural memory ordinary differential equation (nmODE) owns the global attractor theoretically, exhibiting superiority in prompting the model's performance and robustness during applications. While NODE and its variants have been widely used in medical image analysis tasks, there is a lack of a comprehensive review of their applications, hindering the in-depth understanding of NODE's working principle and its potential applications. To mitigate this limitation, this paper thoroughly reviews the literature on the applications of NODE in medical image analysis from the following five aspects: segmentation, reconstruction, registration, disease prediction, and data generation. We also summarize both the strengths and downsides of the applications of NODE, followed by the possible research directions. To the best of our knowledge, this is the first review regards the applications of NODE in the field of medical image analysis. We hope this review can draw the researchers' attention to the great potential of NODE and its variants in medical image analysis.

show abstract

“…On the other hand, a synergistic methodology involves the amalgamation of convolutional neural networks (CNNs) with transfer learning approaches [160]. The essence of this approach is in the use of parameters obtained by convolutional neural networks (CNNs) in the context of the primary application to enable the training of the modal.…”

Section: A Dataset and Labelingmentioning

confidence: 99%

A Comprehensive Review and Analysis of Deep Learning-Based Medical Image Adversarial Attack and Defense

Muoka,

Yi,

Ukwuoma

et al. 2023

Mathematics

View full text Add to dashboard Cite

Deep learning approaches have demonstrated great achievements in the field of computer-aided medical image analysis, improving the precision of diagnosis across a range of medical disorders. These developments have not, however, been immune to the appearance of adversarial attacks, creating the possibility of incorrect diagnosis with substantial clinical implications. Concurrently, the field has seen notable advancements in defending against such targeted adversary intrusions in deep medical diagnostic systems. In the context of medical image analysis, this article provides a comprehensive survey of current advancements in adversarial attacks and their accompanying defensive strategies. In addition, a comprehensive conceptual analysis is presented, including several adversarial attacks and defensive strategies designed for the interpretation of medical images. This survey, which draws on qualitative and quantitative findings, concludes with a thorough discussion of the problems with adversarial attack and defensive mechanisms that are unique to medical image analysis systems, opening up new directions for future research. We identified that the main problems with adversarial attack and defense in medical imaging include dataset and labeling, computational resources, robustness against target attacks, evaluation of transferability and adaptability, interpretability and explainability, real-time detection and response, and adversarial attacks in multi-modal fusion. The area of medical imaging adversarial attack and defensive mechanisms might move toward more secure, dependable, and therapeutically useful deep learning systems by filling in these research gaps and following these future objectives.

show abstract

Adversarially Robust Prototypical Few-Shot Segmentation with Neural-ODEs

Cited by 9 publications

References 21 publications

Holistic Prototype Attention Network for Few-Shot Video Object Segmentation

Holistic Prototype Attention Network for Few-Shot Video Object Segmentation

On the Applications of Neural Ordinary Differential Equations in Medical Image Analysis

A Comprehensive Review and Analysis of Deep Learning-Based Medical Image Adversarial Attack and Defense

Contact Info

Product

Resources

About