HEROHE Challenge: Predicting HER2 Status in Breast Cancer from Hematoxylin–Eosin Whole-Slide Imaging

Abstract

Breast cancer is the most common malignancy in women worldwide, and is responsible for more than half a million deaths each year. The appropriate therapy depends on the evaluation of the expression of various biomarkers, such as the human epidermal growth factor receptor 2 (HER2) transmembrane protein, through specialized techniques, such as immunohistochemistry or in situ hybridization. In this work, we present the HER2 on hematoxylin and eosin (HEROHE) challenge, a parallel event of the 16th European Congress on Digital Pathology, which aimed to predict the HER2 status in breast cancer based only on hematoxylin–eosin-stained tissue samples, thus avoiding specialized techniques. The challenge consisted of a large, annotated, whole-slide images dataset (509), specifically collected for the challenge. Models for predicting HER2 status were presented by 21 teams worldwide. The best-performing models are presented by detailing the network architectures and key parameters. Methods are compared and approaches, core methodologies, and software choices contrasted. Different evaluation metrics are discussed, as well as the performance of the presented models for each of these metrics. Potential differences in ranking that would result from different choices of evaluation metrics highlight the need for careful consideration at the time of their selection, as the results show that some metrics may misrepresent the true potential of a model to solve the problem for which it was developed. The HEROHE dataset remains publicly available to promote advances in the field of computational pathology.

Deep learning to predict the proportion of positive cells in CMYC-stained tissue microarrays of diffuse large B-cell lymphoma

“…The best results achieved accuracies of around 90%. More recently, a similar grand challenge aimed to predict IHC scores from routine H&E whole-slide images [10]. Methods greatly varied in this study and resulted in AUCs ranging from 0.50 to 0.84, similar to a previous study of ours [11].…”

Predicting IHC staining classes of NF1 using features in the hematoxylin channel

“… 6 Most recently, the HEROHE grand challenge was organized to identify high performing machine learning algorithms for prediction of HER2 expression in breast cancer from H&E images. 33 The top-performing teams achieved AUCs of 0.71–0.74, with 1 team reaching an AUC of 0.84. The H&E images and ground-truth IHC and ISH HER2 annotations for each case are publicly available.…”

Future Practices of Breast Pathology Using Digital and Computational Pathology

“…Similar principles have been used to predict HER2 status from tumor morphology on H&E digital slides. 69,70 Tests using machine learning must be extensively validated for such results, as the performance for a test with treatment management decision implications remains quite high. In the early testing phases, it may be prudent to use machine learning decision support systems as an adjunct to molecular testing to validate their performance.…”

“…Shamai et al 68 also developed a deep learning model that predicted ER expression solely from H&E-stained breast pathology images with noninferior accuracy to standard immunohistochemical studies. Similar principles have been used to predict HER2 status from tumor morphology on H&E digital slides 69,70 . Tests using machine learning must be extensively validated for such results, as the performance for a test with treatment management decision implications remains quite high.…”