Abstract 457: Immunohistochemistry-informed AI systems for improved characterization of tumor-microenvironment in clinical non-small cell lung cancer H&E samples

Abstract

Background: Automated cell-level characterization of the tumor microenvironment (TME) at scale is key to data-driven immuno-oncology. Artificial intelligence (AI)-powered analysis of hematoxylin and eosin (H&E) images scales and has recently been translated into diagnostics. However, robust TME analysis solely based on H&E data is bound by the stain's properties and by manual pathologist annotations, both in number and accuracy. In this study, we quantify the error introduced by pathologists' morphological assessment and mitigate this error by training AI-systems without manual pathologist annotations, using labels determined directly from IHC profiles. Methods: The work was carried out on 239 clinical NSCLC cases. CK-KL1, CD3+CD20, and Mum1 were used for defining carcinoma (CA), lymphocyte (LY), and plasma (PL) cells. For evaluation, representative regions were annotated by 3 trained pathologists. The workflow is based on co-registration of same-section H&E and IHC stained images with single cell precision. Cells were detected in H&E and labelled using rule-based algorithms that incorporated IHC information. This H&E data was used to train neural networks (NN). Results: (A) The inter-rater agreement of pathologists annotating on H&E is increased when information from registered IHC images is provided. (B) The concordance of pathologists on H&E-only compared to on H&E+IHC shows that pathologists miss or misclassify cells with a certain error. (C) NNs trained with IHC-based labels achieve similar performance for cell type classification on H&E as pathologists on H&E. Conclusion: This study demonstrates the value of combining histomorphological and IHC data for improved cell annotation. Our novel workflow provides a quantitative benchmark and facilitates training of accurate AI models for quantitative characterization of tumor and TME from H&E sections. A) Inter-rater agreement by metric, stain, and cell type By cell count, Pearson correlation By cell count, Pearson correlation By cell location, Krippendorff’s alpha By cell location, Krippendorff’s alpha Cell type H&E-only H&E+IHC H&E-only H&E+IHC CA 0.86 0.98 0.43 0.90 LY 0.88 0.99 0.21 0.76 PL 0.77 0.96 0.32 0.87 B) Performance of individual pathologists in H&E Against consensus in H&E+IHC Against own annotations in H&E+IHC Against own annotations in H&E+IHC Cell type By cell count, Pearson correlation By cell location, Precision By cell location, Recall CA 0.84 0.76 0.77 LY 0.78 0.70 0.60 PL 0.76 0.69 0.21 C) NN against annotator H&E+IHC consensus Cell Type By cell count, Pearson correlation CA 0.84 LY 0.92 PL 0.75 Citation Format: Thomas Mrowiec, Sharon Ruane, Simon Schallenberg, Gabriel Dernbach, Rumyana Todorova, Cornelius Böhm, Walter de Back, Blanca Pablos, Roman Schulte-Sasse, Ivana Trajanovska, Adelaida Creosteanu, Emil Barbuta, Marcus Otte, Christian Ihling, Hans Juergen Grote, Juergen Scheuenpflug, Viktor Matyas, Maximilian Alber, Frederick Klauschen. Immunohistochemistry-informed AI systems for improved characterization of tumor-microenvironment in clinical non-small cell lung cancer H&E samples [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2022; 2022 Apr 8-13. Philadelphia (PA): AACR; Cancer Res 2022;82(12_Suppl):Abstract nr 457.