OODformer: Out-Of-Distribution Detection Transformer

Koner, Rajat; Sinhamahapatra, Poulami; Roscher, Karsten; Günnemann, Stephan; Tresp, Volker

doi:10.48550/arxiv.2107.08976

Cited by 9 publications

(13 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Extensive research exists on CNN-based out-of-distribution detection approaches in medical imaging [431]- [435]. Recently, few attempts have been made to show that large-scale pretrained ViTs, due to their high-quality representations, can significantly improve the state-of-the-art on a range of out-ofdistribution tasks across different data modalities [386], [430], [436]. However, investigation in these works has been mostly carried out on toy datasets such as CIFAR-10 and CIFAR-100, therefore not necessarily reflecting out-of-distribution detection performance on medical images with complex textures and patterns, high variance in feature scale (like in Xray images), and local specific features.…”

Section: Domain Adaptation and Out-of-distribution Detectionmentioning

confidence: 99%

Transformers in Medical Imaging: A Survey

Shamshad¹,

Khan²,

Zamir³

et al. 2022

Preprint

View full text Add to dashboard Cite

Following unprecedented success on the natural language tasks, Transformers have been successfully applied to several computer vision problems, achieving state-of-the-art results and prompting researchers to reconsider the supremacy of convolutional neural networks (CNNs) as de facto operators. Capitalizing on these advances in computer vision, the medical imaging field has also witnessed growing interest for Transformers that can capture global context compared to CNNs with local receptive fields. Inspired from this transition, in this survey, we attempt to provide a comprehensive review of the applications of Transformers in medical imaging covering various aspects, ranging from recently proposed architectural designs to unsolved issues. Specifically, we survey the use of Transformers in medical image segmentation, detection, classification, reconstruction, synthesis, registration, clinical report generation, and other tasks. In particular, for each of these applications, we develop taxonomy, identify application-specific challenges as well as provide insights to solve them, and highlight recent trends. Further, we provide a critical discussion of the field's current state as a whole, including the identification of key challenges, open problems, and outlining promising future directions. We hope this survey will ignite further interest in the community and provide researchers with an up-to-date reference regarding applications of Transformer models in medical imaging. Finally, to cope with the rapid development in this field, we intend to regularly update the relevant latest papers and their open-source implementations at https://github.com/fahadshamshad/awesome-transformers-in-medical-imaging.

show abstract

Section: Domain Adaptation and Out-of-distribution Detectionmentioning

confidence: 99%

Transformers in Medical Imaging: A Survey

Shamshad¹,

Khan²,

Zamir³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Another line of research assumes that auxiliary data sets are available during the training process. For example, Hendrycks et al ( 2019a ) let classifiers learn OOD example samples before generalizing to other types of OOD data; Mohseni et al ( 2020 ) uses auxiliary data to train OOD detectors; and more recently, Fort et al ( 2021 ) and Koner et al ( 2021 ) fine-tune and improve on the pre-trained Transtormer model.…”

Section: Related Workmentioning

confidence: 99%

Semantic enhanced for out-of-distribution detection

Jiang

2022

Front. Neurorobot.

View full text Add to dashboard Cite

While improving the performance on the out-of-distribution (OOD) benchmark dataset, the existing approach misses a portion of the valid discriminative information such that it reduces the performance on the same manifold OOD (SMOOD) data. The key to addressing this problem is to prompt the model to learn effective and comprehensive in-distribution (ID) semantic features. In this paper, two strategies are proposed to improve the generalization ability of the model to OOD data. Firstly, the original samples are replaced by features extracted from multiple “semantic perspectives” to obtain a comprehensive semantics of the samples; Second, the mean and variance of the batch samples are perturbed in the inference stage to improve the sensitivity of the model to the OOD data. The method we propose does not employ OOD samples, uses no pre-trained models in training, and does not require pre-processing of samples during inference. Experimental results show that our method enhances the semantic representation of ID data, surpasses state-of-the-art detection results on the OOD benchmark dataset, and significantly improves the performance of the model in detecting the SMOOD data.

show abstract

“…Transformer in Vision: In recent times, transformer-based architectures have emerged as the defacto gold standard model for various multi-domain and multi-modal tasks such as image classification [13], object detection [7], and out-of-distribution detection [26]. DETR [7] proposed an end-to-end transformer-based object detection approach with learnable object queries ([obj]-tokens) and direct set-based prediction.…”

Section: Related Workmentioning

confidence: 99%

Relationformer: A Unified Framework for Image-to-Graph Generation

Shit

Koner

Wittmann

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

A comprehensive representation of an image requires understanding objects and their mutual relationship, especially in image-to-graph generation, e.g., road network extraction, blood-vessel network extraction, or scene graph generation. Traditionally, image-to-graph generation is addressed with a two-stage approach consisting of object detection followed by a separate relation prediction, which prevents simultaneous object-relation interaction. This work proposes a unified onestage transformer-based framework, namely Relationformer that jointly predicts objects and their relations. We leverage direct set-based object prediction and incorporate the interaction among the objects to learn an object-relation representation jointly. In addition to existing [obj]-tokens, we propose a novel learnable token, namely [rln]-token. Together with [obj]-tokens, [rln]-token exploits local and global semantic reasoning in an image through a series of mutual associations. In combination with the pair-wise [obj]-token, the [rln]-token contributes to a computationally efficient relation prediction. We achieve state-of-the-art performance on multiple, diverse and multi-domain datasets that demonstrate our approach's effectiveness and generalizability. 2 * equal contribution 2 code is available at https://github.com/suprosanna/relationformer

show abstract

OODformer: Out-Of-Distribution Detection Transformer

Cited by 9 publications

References 32 publications

Transformers in Medical Imaging: A Survey

Transformers in Medical Imaging: A Survey

Semantic enhanced for out-of-distribution detection

Relationformer: A Unified Framework for Image-to-Graph Generation

Contact Info

Product

Resources

About