A review: Deep learning for medical image segmentation using multi-modality fusion

Zhou, Tongxue; Ruan, Su; Canu, Stéphane

doi:10.1016/j.array.2019.100004

Cited by 434 publications

(221 citation statements)

References 29 publications

Supporting

Mentioning

218

Contrasting

Unclassified

Order By: Relevance

“…However, it should be noted that these techniques have been used for single-modal images (MRI or PET) in the majority of the work. Unfortunately, few works have exploited multimodality for the diagnosis of AD like [10,17,46,154,184,186,187,206,207], which seems to be better achieved by serving the advantages of several types of images, each measuring a different type of structural or functional characteristic. In reality, the various imaging environments and modalities offer complementary information that is useful when used in conjunction.…”

Section: Critical Discussion About the Multimodal Diagnosis Of Admentioning

confidence: 99%

See 1 more Smart Citation

A Survey on Computer-Aided Diagnosis of Brain Disorders through MRI Based on Machine Learning and Data Mining Methodologies with an Emphasis on Alzheimer Disease Diagnosis and the Contribution of the Multimodal Fusion

2020

View full text Add to dashboard Cite

Computer-aided diagnostic (CAD) systems use machine learning methods that provide a synergistic effect between the neuroradiologist and the computer, enabling an efficient and rapid diagnosis of the patient’s condition. As part of the early diagnosis of Alzheimer’s disease (AD), which is a major public health problem, the CAD system provides a neuropsychological assessment that helps mitigate its effects. The use of data fusion techniques by CAD systems has proven to be useful, they allow for the merging of information relating to the brain and its tissues from MRI, with that of other types of modalities. This multimodal fusion refines the quality of brain images by reducing redundancy and randomness, which contributes to improving the clinical reliability of the diagnosis compared to the use of a single modality. The purpose of this article is first to determine the main steps of the CAD system for brain magnetic resonance imaging (MRI). Then to bring together some research work related to the diagnosis of brain disorders, emphasizing AD. Thus the most used methods in the stages of classification and brain regions segmentation are described, highlighting their advantages and disadvantages. Secondly, on the basis of the raised problem, we propose a solution within the framework of multimodal fusion. In this context, based on quantitative measurement parameters, a performance study of multimodal CAD systems is proposed by comparing their effectiveness with those exploiting a single MRI modality. In this case, advances in information fusion techniques in medical imagery are accentuated, highlighting their advantages and disadvantages. The contribution of multimodal fusion and the interest of hybrid models are finally addressed, as well as the main scientific assertions made, in the field of brain disease diagnosis.

show abstract

Section: Critical Discussion About the Multimodal Diagnosis Of Admentioning

confidence: 99%

“…Artificial neural networks: ANNs used in several works related to neuroimaging [38][39][40][41][42][43][44][45][46] • Threshold-based techniques: the easiest way is to convert a grayscale image to a binary using a threshold value [68]. Pixels lighter than the threshold are white pixels in the resulting image and darker pixels are black pixels.…”

mentioning

confidence: 99%

A Survey on Computer-Aided Diagnosis of Brain Disorders through MRI Based on Machine Learning and Data Mining Methodologies with an Emphasis on Alzheimer Disease Diagnosis and the Contribution of the Multimodal Fusion

2020

View full text Add to dashboard Cite

show abstract

“…to realize multi-organ segmentation [17] and lesion segmentation [18,19] is widely adopted due to distinct responses of different modalities datasets for different tissues. According to the review [20] of deep learning for medical image segmentation using multi-modality fusion, multi-modal segmentation network architectures can be categorized into input-level fusion network, layer-level fusion network and decision fusion network. The input-level fusion network [21,22] stacks multi-modality images channel-wise and directly feeds them into neural network to make final decisions.…”

Section: Multi-modal Fusionmentioning

confidence: 99%

MMFNet: A multi-modality MRI fusion network for segmentation of nasopharyngeal carcinoma

Huai

Yin

et al. 2020

Neurocomputing

View full text Add to dashboard Cite

Segmentation of nasopharyngeal carcinoma (NPC) from Magnetic ResonanceImages (MRI) is a crucial prerequisite for NPC radiotherapy. However, manually segmenting of NPC is time-consuming and labor-intensive. Additionally, single-modality MRI generally cannot provide enough information for its accurate delineation. Therefore, a multi-modality MRI fusion network (MMFNet), which is a novel framework to fuse information from multi-modality medical images, is proposed to utilize MRI of T1, T2 and contrast-enhanced T1 to complete accurate segmentation of NPC. The backbone of MMFNet is designed as a multi-encoder-based network, consisting of several encoders to capture modality-specific features and one decoder to obtain fused features for NPC segmentation. A fusion block is presented to effectively fuse multi-source features.It contains a 3D Convolutional Block Attention Module (3D-CBAM), recalibrating low-level features captured from modality-specific encoders to highlight both informative features and regions of interest (ROIs), and a residual fusion block (RFBlock), which fuses re-weighted features to keep balance between fused ones and high-level features from decoder. Moreover, in order to make full mining of individual information from multi-modality MRI, a training strategy named self- * transfer is proposed to utilize pre-trained modality-specific encoders to initialize multi-encoder-based network. The proposed method based on multi-modality MRI can effectively segment NPC and its advantages are validated by extensive experiments.

show abstract

“…To overcome this problem, we adopted a novel 3D patch-based approach for training with weighted sampling. Zhou et al (2019) reviewed 2D and 3D patch extraction methods along with several types of loss FIGURE 2 | Patch center localization by randomly selecting x, y, z coordinates in brain volume.…”

Section: Patch Extractionmentioning

confidence: 99%

A Novel Approach for Fully Automatic Intra-Tumor Segmentation With 3D U-Net Architecture for Gliomas

Baid¹,

Talbar²,

Rane

et al. 2020

Front. Comput. Neurosci.

View full text Add to dashboard Cite

Purpose: Gliomas are the most common primary brain malignancies, with varying degrees of aggressiveness and prognosis. Understanding of tumor biology and intra-tumor heterogeneity is necessary for planning personalized therapy and predicting response to therapy. Accurate tumoral and intra-tumoral segmentation on MRI is the first step toward understanding the tumor biology through computational methods. The purpose of this study was to design a segmentation algorithm and evaluate its performance on pre-treatment brain MRIs obtained from patients with gliomas. Materials and Methods:In this study, we have designed a novel 3D U-Net architecture that segments various radiologically identifiable sub-regions like edema, enhancing tumor, and necrosis. Weighted patch extraction scheme from the tumor border regions is proposed to address the problem of class imbalance between tumor and non-tumorous patches. The architecture consists of a contracting path to capture context and the symmetric expanding path that enables precise localization. The Deep Convolutional Neural Network (DCNN) based architecture is trained on 285 patients, validated on 66 patients and tested on 191 patients with Glioma from Brain Tumor Segmentation (BraTS) 2018 challenge dataset. Three dimensional patches are extracted from multi-channel BraTS training dataset to train 3D U-Net architecture. The efficacy of the proposed approach is also tested on an independent dataset of 40 patients with High Grade Glioma from our tertiary cancer center. Segmentation results are assessed in terms of Dice Score, Sensitivity, Specificity, and Hausdorff 95 distance (ITCN intra-tumoral classification network).Result: Our proposed architecture achieved Dice scores of 0.88, 0.83, and 0.75 for the whole tumor, tumor core and enhancing tumor, respectively, on BraTS validation dataset and 0.85, 0.77, 0.67 on test dataset. The results were similar on the independent patients' dataset from our hospital, achieving Dice scores of 0.92, 0.90, and 0.81 for the whole tumor, tumor core and enhancing tumor, respectively. Baid et al. Automatic Intra-Tumor Segmentation for GliomasConclusion: The results of this study show the potential of patch-based 3D U-Net for the accurate intra-tumor segmentation. From experiments, it is observed that the weighted patch-based segmentation approach gives comparable performance with the pixel-based approach when there is a thin boundary between tumor subparts.

show abstract

A review: Deep learning for medical image segmentation using multi-modality fusion

Cited by 434 publications

References 29 publications

A Survey on Computer-Aided Diagnosis of Brain Disorders through MRI Based on Machine Learning and Data Mining Methodologies with an Emphasis on Alzheimer Disease Diagnosis and the Contribution of the Multimodal Fusion

A Survey on Computer-Aided Diagnosis of Brain Disorders through MRI Based on Machine Learning and Data Mining Methodologies with an Emphasis on Alzheimer Disease Diagnosis and the Contribution of the Multimodal Fusion

MMFNet: A multi-modality MRI fusion network for segmentation of nasopharyngeal carcinoma

A Novel Approach for Fully Automatic Intra-Tumor Segmentation With 3D U-Net Architecture for Gliomas

Contact Info

Product

Resources

About