Glioma Classification Using Multimodal Radiology and Histology Data

Hamidinekoo, Azam; Pieciak, Tomasz; Afzali, Maryam; Akanyeti, Otar; Yuan, Yinyin

doi:10.1007/978-3-030-72087-2_45

Cited by 5 publications

(7 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The performance of these methods is affected by the segmentation results. Methods in ( Hamidinekoo et al, 2020 ; Lerousseau et al, 2020 ) and our solutions do not require the segmentation process.…”

Section: Resultsmentioning

confidence: 99%

“…Unlike previous studies, automatic classification methods based on multimodal brain images have been recently investigated. These related works mainly came out of the MICCAI 2019 and 2020 CPM-RadPath Challenge ( Chan et al, 2019 ; Pei et al, 2019 ; Hamidinekoo et al, 2020 ; Lerousseau et al, 2020 ; Pei et al, 2020 ; Yin et al, 2020 ; Zhao et al, 2020 ). Based on the CPM-RadPath data, we expect to propose an accurate automatic classification method based on multimodal data.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Combining Radiology and Pathology for Automatic Glioma Classification

Wang

Yang

et al. 2022

Front. Bioeng. Biotechnol.

View full text Add to dashboard Cite

Subtype classification is critical in the treatment of gliomas because different subtypes lead to different treatment options and postoperative care. Although many radiological- or histological-based glioma classification algorithms have been developed, most of them focus on single-modality data. In this paper, we propose an innovative two-stage model to classify gliomas into three subtypes (i.e., glioblastoma, oligodendroglioma, and astrocytoma) based on radiology and histology data. In the first stage, our model classifies each image as having glioblastoma or not. Based on the obtained non-glioblastoma images, the second stage aims to accurately distinguish astrocytoma and oligodendroglioma. The radiological images and histological images pass through the two-stage design with 3D and 2D models, respectively. Then, an ensemble classification network is designed to automatically integrate the features of the two modalities. We have verified our method by participating in the MICCAI 2020 CPM-RadPath Challenge and won 1st place. Our proposed model achieves high performance on the validation set with a balanced accuracy of 0.889, Cohen’s Kappa of 0.903, and an F1-score of 0.943. Our model could advance multimodal-based glioma research and provide assistance to pathologists and neurologists in diagnosing glioma subtypes. The code has been publicly available online at https://github.com/Xiyue-Wang/1st-in-MICCAI2020-CPM.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Combining Radiology and Pathology for Automatic Glioma Classification

Wang

Yang

et al. 2022

Front. Bioeng. Biotechnol.

View full text Add to dashboard Cite

show abstract

“…The data extraction stage was followed by fusion and classification to predict disease diagnosis for telemedicine. Most deep learning feature extraction strategies apply CNNbased models for retrieving features from various data modalities (Hilmizen et al, 2020;Carvalho et al, 2021;Hamidinekoo et al, 2021). Table 4 shows an overview of various multimodal medical data extraction techniques.…”

Section: Multimodal Medical Data Extractionmentioning

confidence: 99%

“…proposed a late fusion strategy including summation, ranking and multiplication to fuse unstructured features extracted from patient questionnaires and structured symptom data. Features extracted are separately passed through various machine learning models before being merged for prediction Hamidinekoo et al (2021). showcased a fusion of a deep convolutional network (DCN) feature obtained from MRI and whole slide imaging (WSI) pictures using a late fusion technique including majority voting.…”

mentioning

confidence: 99%

Comprehensive Review of Multimodal Medical Data Analysis: Open Issues and Future Research Directions

Shetty¹,

Ananthanarayana²,

Mahale³

2022

AIP

View full text Add to dashboard Cite

Over the past few decades, the enormous expansion of medical data has led to searching for ways of data analysis in smart healthcare systems. Acquisition of data from pictures, archives, communication systems, electronic health records, online documents, radiology reports and clinical records of different styles with specific numerical information has given rise to the concept of multimodality and the need for machine learning and deep learning techniques in the analysis of the healthcare system. Medical data play a vital role in medical education and diagnosis; determining dependency between distinct modalities is essential. This paper gives a gist of current radiology medical data analysis techniques and their various approaches and frameworks for representation and classification. A brief outline of the existing medical multimodal data processing work is presented. The main objective of this study is to spot gaps in the surveyed area and list future tasks and challenges in radiology. The Preferred Reporting Items for Systematic Reviews and Meta-Analysis (or PRISMA) guidelines were incorporated in this study for effective article search and to investigate several relevant scientific publications. The systematic review was carried out on multimodal medical data analysis and highlighted advantages, limitations and strategies. The inherent benefit of multimodality in the medical domain powered with artificial intelligence has a significant impact on the performance of the disease diagnosis frameworks.

show abstract

“…However, because nasopharyngeal carcinomas vary in terms of shape, size, and internal characteristics, the low-level handcrafted features of traditional computer-aided diagnosis methods are inadequate. Under such circumstances, deep learning can be very helpful since its features are learned both automatically and effectively (Hamidinekoo et al 2020, Zhuge et al 2020, Chang et al 2021, Xie et al 2021. A convolutional neural network (CNN)-based method has recently been used for nasopharyngeal carcinoma Tstaging that involves only two-dimensional (2D) images (Yang et al 2020).…”

Section: Introductionmentioning

confidence: 99%

A multi-perspective information aggregation network for automated T-staging detection of nasopharyngeal carcinoma

Liang¹,

Dong²,

Yang³

et al. 2022

Phys. Med. Biol.

View full text Add to dashboard Cite

Accurate T-staging is important when planning personalized radiotherapy. However, T-staging via manual slice-by-slice inspection is time-consuming while tumor sizes and shapes are heterogeneous, and junior physicians find such inspection challenging. With inspiration from oncological diagnostics, we developed a multi-perspective aggregation network that incorporated various diagnosis-oriented knowledge which allowed automated nasopharyngeal carcinoma T-staging detection (TSD Net). Specifically, our TSD Net was designed in multi-branch architecture, which can capture tumor size and shape information (basic knowledge), strongly correlated contextual features, and associations between the tumor and surrounding tissues. We defined the association between the tumor and surrounding tissues by a signed distance map (SDM) which can embed points and tumor contours in higher-dimensional spaces, yielding valuable information regarding the locations of tissue associations. TSD Net finally outputs a T1-T4 stage prediction by aggregating data from the three branches. We evaluated TSD Net by using the T1-weighted contrast-enhanced (T1CE) magnetic resonance imaging database of 320 patients in three-fold cross-validations manner. The results show that the proposed method achieves a mean area under the curve (AUC) as high as 87.95%. We also compared our method to traditional classifiers and a deep learning-based method. Our TSD Net is efficient and accurate and outperforms other methods.

show abstract

Glioma Classification Using Multimodal Radiology and Histology Data

Cited by 5 publications

References 16 publications

Combining Radiology and Pathology for Automatic Glioma Classification

Combining Radiology and Pathology for Automatic Glioma Classification

Comprehensive Review of Multimodal Medical Data Analysis: Open Issues and Future Research Directions

A multi-perspective information aggregation network for automated T-staging detection of nasopharyngeal carcinoma

Contact Info

Product

Resources

About