FDTrans: Frequency Domain Transformer Model for predicting subtypes of lung cancer using multimodal data

Cai, Meiling; Zhao, Lin; Hou, Guojie; Zhang, Yanan; Wu, Wei; Jia, Liye; Zhao, Juanjuan; Wang, Long; Qiang, Yan

doi:10.1016/j.compbiomed.2023.106812

Cited by 5 publications

(3 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…AMIGO [ 3 ] created a multi-modal graph transformer architecture that predicts patient survival based on multi-modal histopathological images and shared related data. Cai et al [ 60 ] created a frequency-domain transformer architecture that integrates frequency and spatial domains for histopathological lung cancer image analysis and subtype determination.…”

Section: Discussionmentioning

confidence: 99%

“…For breast cancer histopathological image classification, DCET-Net [ 72 ] proposed a dual-stream convolution-expanded transformer architecture; Breast-Net [ 51 ] explores the ability of ensemble learning techniques using four Swin transformer architectures; HATNet [ 52 ] uses end-to-end vision transformers with a self-attention mechanism; ScoreNet [ 16 ] developed an efficient transformer-based architecture that integrates a coarse-grained global attention framework with a fine-grained local attention mechanism framework; LGVIT [ 73 ] built a local–global ViT model by introducing a new local–global MHSA mechanism and a ghost geed-forward network block into the network; dMIL-transformer [ 53 ] developed a two-stage double max–min multiple-instance learning (MIL) transformer architecture that combines both the spatial and morphological information of the cancer regions. Other than breast cancer classification, transformers have also been applied to other histopathological image cancer classification tasks, such as bone cancer classification (NRCA-FCFL [ 74 ]), brain cancer classification (ViT-WSI [ 17 ], ASI-DBNet [ 54 ], Ding et al [ 55 ]), colorectal cancer classification (MIST [ 75 ], DT-DSMIL [ 56 ]), gastric cancer classification (IMGL-VTNet [ 57 ]), kidney subtype classification (i-ViT [ 59 ], tRNAsformer [ 58 ]), thymoma or thymic carcinoma classification (MC-ViT [ 76 ]), lung cancer classification (GTP [ 46 ], FDTrans [ 60 ]), skin cancer classification (Wang et al [ 45 ]), and thyroid cancer classification (Wang et al [ 77 ], PyT2T-ViT [ 41 ], Wang et al [ 78 ]) using different transformer-based architectures. Furthermore, other transformer models such as Transmil [ 65 ], KAT [ 61 ], ViT-based unsupervised contrastive learning architecture [ 79 ], DecT [ 66 ], StoHisNet [ 80 ], CWC-transformer [ 63 ], LA-MIL [ 44 ], SETMIL [ 81 ], Prompt-MIL [ 67 ], GLAMIL [ 67 ], MaskHIT [ 82 ], HAG-MIL [ 68 ], MEGT [ 47 ], MSPT [ 70 ], and HistPathGPT [ 69 ] have also been evaluated on more than one tissue type, such as liver, prostate, breast, brain, gastric, kidney, lung, colorectal, and so on, for h...…”

Section: Current Progressmentioning

confidence: 99%

See 1 more Smart Citation

A survey of Transformer applications for histopathological image analysis: New developments and future directions

Atabansi,

Nie,

Liu

et al. 2023

BioMed Eng OnLine

View full text Add to dashboard Cite

Transformers have been widely used in many computer vision challenges and have shown the capability of producing better results than convolutional neural networks (CNNs). Taking advantage of capturing long-range contextual information and learning more complex relations in the image data, Transformers have been used and applied to histopathological image processing tasks. In this survey, we make an effort to present a thorough analysis of the uses of Transformers in histopathological image analysis, covering several topics, from the newly built Transformer models to unresolved challenges. To be more precise, we first begin by outlining the fundamental principles of the attention mechanism included in Transformer models and other key frameworks. Second, we analyze Transformer-based applications in the histopathological imaging domain and provide a thorough evaluation of more than 100 research publications across different downstream tasks to cover the most recent innovations, including survival analysis and prediction, segmentation, classification, detection, and representation. Within this survey work, we also compare the performance of CNN-based techniques to Transformers based on recently published papers, highlight major challenges, and provide interesting future research directions. Despite the outstanding performance of the Transformer-based architectures in a number of papers reviewed in this survey, we anticipate that further improvements and exploration of Transformers in the histopathological imaging domain are still required in the future. We hope that this survey paper will give readers in this field of study a thorough understanding of Transformer-based techniques in histopathological image analysis, and an up-to-date paper list summary will be provided at https://github.com/S-domain/Survey-Paper.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Current Progressmentioning

confidence: 99%

A survey of Transformer applications for histopathological image analysis: New developments and future directions

Atabansi,

Nie,

Liu

et al. 2023

BioMed Eng OnLine

View full text Add to dashboard Cite

show abstract

“…Despite the progress made by existing methods, challenges remain due to limited annotated datasets, large intra-class differences, and high inter-class similarities. To address these challenges, Cai et al [70] proposed a dual-branch deep learning model called the Frequency Domain Transformer Model (FDTrans). FDTrans combines image domain and genetic information to determine lung cancer subtypes in patients.…”

Section: Deep Learning Techniques For Lung Cancer Using Tcga Datasetmentioning

confidence: 99%

Deep Machine Learning for Medical Diagnosis, Application to Lung Cancer Detection: A Review

Gayap,

Akhloufi

2024

BioMedInformatics

View full text Add to dashboard Cite

Deep learning has emerged as a powerful tool for medical image analysis and diagnosis, demonstrating high performance on tasks such as cancer detection. This literature review synthesizes current research on deep learning techniques applied to lung cancer screening and diagnosis. This review summarizes the state-of-the-art in deep learning for lung cancer detection, highlighting key advances, limitations, and future directions. We prioritized studies utilizing major public datasets, such as LIDC, LUNA16, and JSRT, to provide a comprehensive overview of the field. We focus on deep learning architectures, including 2D and 3D convolutional neural networks (CNNs), dual-path networks, Natural Language Processing (NLP) and vision transformers (ViT). Across studies, deep learning models consistently outperformed traditional machine learning techniques in terms of accuracy, sensitivity, and specificity for lung cancer detection in CT scans. This is attributed to the ability of deep learning models to automatically learn discriminative features from medical images and model complex spatial relationships. However, several challenges remain to be addressed before deep learning models can be widely deployed in clinical practice. These include model dependence on training data, generalization across datasets, integration of clinical metadata, and model interpretability. Overall, deep learning demonstrates great potential for lung cancer detection and precision medicine. However, more research is required to rigorously validate models and address risks. This review provides key insights for both computer scientists and clinicians, summarizing progress and future directions for deep learning in medical image analysis.

show abstract

Deep learning-based multimodal spatial transcriptomics analysis for cancer

Rajdeo,

Aronow,

Surya Prasath

2024

Advances in Cancer Research

View full text Add to dashboard Cite

FDTrans: Frequency Domain Transformer Model for predicting subtypes of lung cancer using multimodal data

Cited by 5 publications

References 28 publications

A survey of Transformer applications for histopathological image analysis: New developments and future directions

A survey of Transformer applications for histopathological image analysis: New developments and future directions

Deep Machine Learning for Medical Diagnosis, Application to Lung Cancer Detection: A Review

Deep learning-based multimodal spatial transcriptomics analysis for cancer

Contact Info

Product

Resources

About