Learning Representations with Contrastive Self-Supervised Learning for Histopathology Applications

Abstract

Unsupervised learning has made substantial progress over the last few years, especially by means of contrastive self-supervised learning. The dominating dataset for benchmarking self-supervised learning has been ImageNet, for which recent methods are approaching the performance achieved by fully supervised training. The ImageNet dataset is however largely object-centric, and it is not clear yet what potential those methods have on widely different datasets and tasks that are not object-centric, such as in digital pathology. While self-supervised learning has started to be explored within this area with encouraging results, there is reason to look closer at how this setting differs from natural images and ImageNet. In this paper we make an in-depth analysis of contrastive learning for histopathology, pinpointing how the contrastive objective will behave differently due to the characteristics of histopathology data. We bring forward a number of considerations, such as view generation for the contrastive objective and hyper-parameter tuning. In a large battery of experiments, we analyze how the downstream performance in tissue classification will be affected by these considerations. The results point to how contrastive learning can reduce the annotation effort within digital pathology, but that the specific dataset characteristics need to be considered. To take full advantage of the contrastive learning objective, different calibrations of view generation and hyper-parameters are required. Our results pave the way for realizing the full potential of self-supervised learning for histopathology applications. Code and trained models are available at https://github.com/k-stacke/ssl-pathology.

Contrastive Multiple Instance Learning: An Unsupervised Framework for Learning Slide-Level Representations of Whole Slide Histopathology Images without Labels

“…Finally, they showed that combining image patches from different sites did not improve downstream performance and was comparable to single-site performance. The same conclusions were drawn earlier by Stacke et al [ 23 ], who applied SimCLR to patch-wise breast cancer and skin cancer classification. They also showed that the set of optimal transformations changed depending on the dataset utilized.…”

“…These works define pretext tasks from which patch-wise feature representations are learned. Such pretext tasks include contrastive predictive coding [ 21 ], contrastive learning on adjacent image patches [ 22 ], contrastive learning using SimCLR [ 23 , 24 , 25 ], and SimSiam [ 26 ] with an additional stop-gradient for adjacent patches [ 27 ]. Many methods utilize generic features derived from ImageNet as their patch-wise feature representations [ 17 , 18 , 28 ].…”

“…This result is not reflected in general-purpose self-supervised learning, where more images result in higher downstream performance. Stacke et al [ 23 ] propose that this may be due to the redundancy in WSI patches (i.e., anatomies tend to be repeated). Thirdly, fewer labels are required when compared to training from scratch or transfer learning from ImageNet weights.…”

Self-distillation Augmented Masked Autoencoders for Histopathological Image Classification

“…It is not easy to extract discriminative feature. Second, the category number of pathological images are typical few, e.g., 2 or 4 [19,26,3]. It is limited compared to natural image understanding tasks, which restrict the learning effectiveness.…”

Artificial intelligence to identify genetic alterations in conventional histopathology

“…Most recently, transformers and graph neural networks that are intrinsically trained with the correlation information between different tiles along with the tile images have been proposed [134,[141][142][143]. Another approach that is becoming more and more common in DL systems in histopathology is contrastive SSL, a subset of unsupervised learning [56,57,144,145]. In contrastive self-supervised training, the model learns the patterns in a dataset in the absence of any labels by contrasting different images and rewarding similar images; therefore, it is aimed to obtain better representations of images [146,147].…”