Adaptive Multimodal Fusion With Attention Guided Deep Supervision Net for Grading Hepatocellular Carcinoma

Li, Shangxuan; Xie, Yanyan; Wang, Guangyi; Zhang, Lijuan; Zhou, Wu

doi:10.1109/jbhi.2022.3161466

Cited by 15 publications

(4 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The cross-modal intra- and inter-attention module considers the intra- and inter-relation among modalities to harness the complementary information among different phases, which can help deep learning focus on the most informative region of each modality and select the most useful features for predictions [ 23 ]. Moreover, attention modules can provide attention weights that can be visualized and display salient regions to predictions, increasing the clinical interpretability of prediction models [ 21 , 24 ]. Therefore, the motivation of this study was to apply the attention-guided feature fusion network proposed by Li’s group [ 23 ] to develop MVI prediction models based on multi-phase MRI and to acquire visual explanations of predictions through the visualization of weights offered by attention modules.…”

Section: Methodsmentioning

confidence: 99%

“…The cros modal intra-and inter-attention module considers the intra-and inter-relation among m dalities to harness the complementary information among different phases, which ca help deep learning focus on the most informative region of each modality and select th most useful features for predictions [23]. Moreover, attention modules can provide atte tion weights that can be visualized and display salient regions to predictions, increasin the clinical interpretability of prediction models [21,24]. Therefore, the motivation of th…”

Section: Model Developmentmentioning

confidence: 99%

“…Furthermore, the weights provided by the attention modules are analytical and easily interpretable [ 21 ]. The visualization of attention weights can directly display salient regions to the final prediction, which offers insights into prediction models and improves its clinical interpretation [ 21 , 24 ]. This suggests that embedding attention mechanisms into deep learning may improve the prediction performance of MVI and reveal the underlying mechanism behind predictions, overcoming the black-box nature of deep learning.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Clinical Interpretability of Deep Learning for Predicting Microvascular Invasion in Hepatocellular Carcinoma by Using Attention Mechanism

You,

Wang,

et al. 2023

Bioengineering

View full text Add to dashboard Cite

Preoperative prediction of microvascular invasion (MVI) is essential for management decision in hepatocellular carcinoma (HCC). Deep learning-based prediction models of MVI are numerous but lack clinical interpretation due to their “black-box” nature. Consequently, we aimed to use an attention-guided feature fusion network, including intra- and inter-attention modules, to solve this problem. This retrospective study recruited 210 HCC patients who underwent gadoxetate-enhanced MRI examination before surgery. The MRIs on pre-contrast, arterial, portal, and hepatobiliary phases (hepatobiliary phase: HBP) were used to develop single-phase and multi-phase models. Attention weights provided by attention modules were used to obtain visual explanations of predictive decisions. The four-phase fusion model achieved the highest area under the curve (AUC) of 0.92 (95% CI: 0.84–1.00), and the other models proposed AUCs of 0.75–0.91. Attention heatmaps of collaborative-attention layers revealed that tumor margins in all phases and peritumoral areas in the arterial phase and HBP were salient regions for MVI prediction. Heatmaps of weights in fully connected layers showed that the HBP contributed the most to MVI prediction. Our study firstly implemented self-attention and collaborative-attention to reveal the relationship between deep features and MVI, improving the clinical interpretation of prediction models. The clinical interpretability offers radiologists and clinicians more confidence to apply deep learning models in clinical practice, helping HCC patients formulate personalized therapies.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Model Developmentmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Clinical Interpretability of Deep Learning for Predicting Microvascular Invasion in Hepatocellular Carcinoma by Using Attention Mechanism

You,

Wang,

et al. 2023

Bioengineering

View full text Add to dashboard Cite

show abstract

“…Cancer probability can be assessed by using the recognized features and their fusion. But, this task can be highly challenging, even for medical experts, because nodule presence and positive cancer diagnoses are not simply interrelated [9]. A computer-aided diagnoses (CAD) approach uses earlier analyzed features that are in some way associated with cancer suspicion, like shape, sphericity, volume, subtlety, speculation, solidity, etc.…”

Section: Introductionmentioning

confidence: 99%

Dung Beetle Optimization with Deep Feature Fusion Model for Lung Cancer Detection and Classification

Alamgeer

Alruwais

Alshahrani

et al. 2023

Cancers

View full text Add to dashboard Cite

Lung cancer is the main cause of cancer deaths all over the world. An important reason for these deaths was late analysis and worse prediction. With the accelerated improvement of deep learning (DL) approaches, DL can be effectively and widely executed for several real-world applications in healthcare systems, like medical image interpretation and disease analysis. Medical imaging devices can be vital in primary-stage lung tumor analysis and the observation of lung tumors from the treatment. Many medical imaging modalities like computed tomography (CT), chest X-ray (CXR), molecular imaging, magnetic resonance imaging (MRI), and positron emission tomography (PET) systems are widely analyzed for lung cancer detection. This article presents a new dung beetle optimization modified deep feature fusion model for lung cancer detection and classification (DBOMDFF-LCC) technique. The presented DBOMDFF-LCC technique mainly depends upon the feature fusion and hyperparameter tuning process. To accomplish this, the DBOMDFF-LCC technique uses a feature fusion process comprising three DL models, namely residual network (ResNet), densely connected network (DenseNet), and Inception-ResNet-v2. Furthermore, the DBO approach was employed for the optimum hyperparameter selection of three DL approaches. For lung cancer detection purposes, the DBOMDFF-LCC system utilizes a long short-term memory (LSTM) approach. The simulation result analysis of the DBOMDFF-LCC technique of the medical dataset is investigated using different evaluation metrics. The extensive comparative results highlighted the betterment of the DBOMDFF-LCC technique of lung cancer classification.

show abstract

CARL: Cross-Aligned Representation Learning for Multi-view Lung Cancer Histology Classification

Luo,

Liu,

Fang

et al. 2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Adaptive Multimodal Fusion With Attention Guided Deep Supervision Net for Grading Hepatocellular Carcinoma

Cited by 15 publications

References 30 publications

Clinical Interpretability of Deep Learning for Predicting Microvascular Invasion in Hepatocellular Carcinoma by Using Attention Mechanism

Clinical Interpretability of Deep Learning for Predicting Microvascular Invasion in Hepatocellular Carcinoma by Using Attention Mechanism

Dung Beetle Optimization with Deep Feature Fusion Model for Lung Cancer Detection and Classification

CARL: Cross-Aligned Representation Learning for Multi-view Lung Cancer Histology Classification

Contact Info

Product

Resources

About