MPMR: Multi-Scale Feature and Probability Map for Melanoma Recognition

Abstract

Malignant melanoma (MM) recognition in whole-slide images (WSIs) is challenging due to the huge image size of billions of pixels and complex visual characteristics. We propose a novel automatic melanoma recognition method based on the multi-scale features and probability map, named MPMR. First, we introduce the idea of breaking up the WSI into patches to overcome the difficult-to-calculate problem of WSIs with huge sizes. Second, to obtain and visualize the recognition result of MM tissues in WSIs, a probability mapping method is proposed to generate the mask based on predicted categories, confidence probabilities, and location information of patches. Third, considering that the pathological features related to melanoma are at different scales, such as tissue, cell, and nucleus, and to enhance the representation of multi-scale features is important for melanoma recognition, we construct a multi-scale feature fusion architecture by additional branch paths and shortcut connections, which extracts the enriched lesion features from low-level features containing more detail information and high-level features containing more semantic information. Fourth, to improve the extraction feature of the irregular-shaped lesion and focus on essential features, we reconstructed the residual blocks by a deformable convolution and channel attention mechanism, which further reduces information redundancy and noisy features. The experimental results demonstrate that the proposed method outperforms the compared algorithms, and it has a potential for practical applications in clinical diagnosis.

Diagnostic and Prognostic Deep Learning Applications for Histological Assessment of Cutaneous Melanoma

“…However, multiple studies show applications in the differentiation between melanoma, nevi, and normal skin [ 34 , 35 ]; and differentiation between melanoma and nonmelanoma skin cancers [ 36 , 37 , 38 ]. Several studies showed deep-learning applications for the segmentation of whole tumor regions [ 39 , 40 , 41 , 42 ] or individual diagnostic markers such as mitotic cells [ 43 , 44 ], melanocytes [ 45 , 46 ], and melanocytic nests [ 47 ]. Several of these models were compared against the diagnostic accuracy of trained histopathologists, showing improved performance [ 25 , 27 , 29 , 31 ] ( Table 1 ).…”

STNet: shape and texture joint learning through two-stream network for knowledge-guided image recognition

“…In the design of the shape encoder network, we introduce the deformable kernel to address the limitation of the rectangular receptive field of the convolution kernel. Irregular-shaped visual features are common in lesion images, for example, the irregular-shape boundary of the lesion in dermoscopic images (Celebi et al, 2019 ), the irregular-shaped cells in pathological images (Zhang D. et al, 2022 ). Rectangular convolutional kernels have limitation in extracting these features, especially in extracting low-level shape features.…”

“…We compared the proposed method with some popular general vision models, including the ResNeSt (Zhang H. et al, 2022 ), which is the latest iteration of ResNet, and ConvNeXt (Liu et al, 2022 ), which is regarded as CNN for 2020s. We also added some models designed for specific medical image recognition tasks to the comparative experiment, including DeMAL-CNN (He et al, 2022 ) for skin lesion classification in dermoscopy images, and MPMR (Zhang D. et al, 2022 ), which is a multi-scale-feature-based melanoma recognition method in pathological images.…”