Convolutional Networks and Transformers for Mammography Classification: An Experimental Study

Abstract

Convolutional Neural Networks (CNN) have received a large share of research in mammography image analysis due to their capability of extracting hierarchical features directly from raw data. Recently, Vision Transformers are emerging as viable alternative to CNNs in medical imaging, in some cases performing on par or better than their convolutional counterparts. In this work, we conduct an extensive experimental study to compare the most recent CNN and Vision Transformer architectures for whole mammograms classification. We selected, trained and tested 33 different models, 19 convolutional- and 14 transformer-based, on the largest publicly available mammography image database OMI-DB. We also performed an analysis of the performance at eight different image resolutions and considering all the individual lesion categories in isolation (masses, calcifications, focal asymmetries, architectural distortions). Our findings confirm the potential of visual transformers, which performed on par with traditional CNNs like ResNet, but at the same time show a superiority of modern convolutional networks like EfficientNet.

Deep Learning in Breast Cancer Imaging: State of the Art and Recent Advancements in Early 2024

“…Recently, Ayana et al developed a ViT-based mammography classifier adapted by transfer learning from a pre-trained model, which was able to achieve better performance than state-of-the-art CNN-based classifiers [142]. However, in a comparison study published in late 2023, Cantone et al observed that ViTbased models may deliver worse mammography classification performance than CNNs when trained with small datasets [143].…”

“…Overall, ConvNeXt has become one of the most competitive computer vision classification models currently available. However, despite being less complex than ViTs, it is still more computationally expensive and harder to train than CNNs, while not always delivering significantly better performance [143].…”

Mammography Datasets for Neural Networks—Survey

“…An example of an article that documents the use of the OMI-DB dataset is the work by Cantone et al [27], which compares Deep Convolutional Neural Networks (DCNN) with Vision Transformers (ViT). The research showed that some ViT architectures can achieve comparable results to traditional DCNN architectures.…”