Multi-View Feature Fusion Based Four Views Model for Mammogram Classification Using Convolutional Neural Network

Abstract

Breast cancer is the second most common cause of cancer-related deaths among women. Early detection leads to better prognosis and saves lives. The 5-year survival rate of breast cancer is 99% if it is located only in breast. Conventional computer-aided diagnosis (CADx) systems for breast cancer use the single view information of mammograms to assist the radiologists. More recent work has focused on more than one views. Existing multi-view based CADx systems normally employ only two views namely Cranio-Caudal (CC) and Medio-Lateral-Oblique (MLO). The information fusion of the two views proved the effectiveness of the system for mammogram classification which cannot be achieved by single view information. However, combining the information of four views of mammograms increases the performance of classification. In this study, we propose Multi-View Feature Fusion (MVFF) based CADx system using feature fusion technique of four views for classification of mammogram. The complete CADx tool contains three stages, the first stage have the ability to classify mammogram into abnormal or normal, second stage is about classification of mass or calcification and in the final stage classification of malignant or benign classification is performed. Convolutional Neural Network (CNN) based feature extraction models operate on each view separately. These extracted features were fused into one final layer for ultimate prediction. Our proposed system is trained on four views of mammograms, after data augmentation. We performed our experiments on publicly available datasets such as CBIS-DDSM (Curated Breast Imaging Subset of DDSM) and mini-MIAS database of mammograms. In comparison with literature the MVFF based system is performed better than a single view-based system for mammogram classification. We have achieved area under ROC curve (AUC) of 0.932 for mass and calcification and 0.84 for malignant and benign, which is higher than all single-view based systems. The value of AUC for normal and abnormal classification is 0.93.

New convolutional neural network model for screening and diagnosis of mammograms

“…Firstly, multi-view CNN is used to fuse two-view image features, and then multi-detail CNN is used to extract deep features. Khan et al [23] constructed a four-view classification network, which divides mammogram patches into normal and abnormal, mass and calcification, benign and malignant in three stages.…”

“…So some researchers have tried to use smaller image patches to classify the lesions. Khan et al [23] used four image patches from the left and right breasts to classify the mammograms in three stages. The first stage is to divide the images into normal and abnormal, the second stage is to divide the abnormal images into mass and calcification, and the third stage is to divide the lesions into benign and malignant.…”

Breast Mass Classification Using Diverse Contextual Information and Convolutional Neural Network

“…In this study, motivated by the related work, we adopted ResNet-50 as a backbone model to classify mammogram mass ROIs as benign or malignant [ 8 , 10 ] and also modified it to adapt it to breast density. Its architecture is based on residual theory, which allows increasing the depth of a CNN model without suffering from degradation problems [ 32 ].…”

“…We focused on the second problem of CAD, i.e., classifying the mass regions into benign or malignant, which is a very difficult problem to resolve [ 5 ]. Recently, medical imaging researchers used innovative deep learning methods to overcome this problem, but their performance is low and may not be acceptable for clinical use [ 6 , 7 , 8 ]. An ensemble classifier can be used for better performance because an ensemble classifier strategy achieves a more promising performance than using a single classifier.…”

“…Khan et al [ 8 ] proposed a three-stage CAD system based on Multi-View Feature Fusion (MVFF), which takes four mammography views as input, extracts CNN features from four views, fuses them using concatenation, and finally classifies them into normal/abnormal, or mass/calcification, and benign/malignant. Although this method uses very rich information as input, its performance (sensitivity of 81.82%, and specificity of 72.02%) for benign and malignant classification is very poor.…”