Evaluating stability of histomorphometric features across scanner and staining variations: prostate cancer diagnosis from whole slide images

Abstract

Abstract. Quantitative histomorphometry (QH) is the process of computerized feature extraction from digitized tissue slide images to predict disease presence, behavior, and outcome. Feature stability between sites may be compromised by laboratory-specific variables including dye batch, slice thickness, and the whole slide scanner used. We present two new measures, preparation-induced instability score and latent instability score, to quantify feature instability across and within datasets. In a use case involving prostate cancer, we examined QH features which may detect cancer on whole slide images. Using our method, we found that five feature families (graph, shape, co-occurring gland tensor, sub-graph, and texture) were different between datasets in 19.7% to 48.6% of comparisons while the values expected without site variation were 4.2% to 4.6%. Color normalizing all images to a template did not reduce instability. Scanning the same 34 slides on three scanners demonstrated that Haralick features were most substantively affected by scanner variation, being unstable in 62% of comparisons. We found that unstable feature families performed significantly worse in inter-than intrasite classification. Our results appear to suggest QH features should be evaluated across sites to assess robustness, and class discriminability alone should not represent the benchmark for digital pathology feature selection.

Nuclear shape and orientation features from H&E images predict survival in early-stage estrogen receptor-positive breast cancers

“…Also, since the patient image data originated from a single facility as opposed to multiple facilities, typical confounding variables such as tissue staining quality differences and differing patient population characteristics were not considered. This batch effect could potentially affect the image and feature analysis steps and will need to be rigorously investigated in future work [31]. Another limitation was that we did not explicitly distinguish cancer cells from tumor-infiltrating lymphocytes (TILs).…”

Stable and discriminating features are predictive of cancer presence and Gleason grade in radical prostatectomy specimens: a multi-site study

“…In this work we employed the stability metric introduced by us in Leo et al . 19 to evaluate feature robustness across different sites. Specifically we evaluated preparation-induced instability score, a measure of the observed difference between feature value distributions between sites, and employed it for improving cross-site classification performance in cancer detection and grading tasks.…”

“…Leo et al . 19 introduced two measures of feature stability, latent instability (LI) and preparation-induced instability (PI). LI captures the instability from differences in patient population in the absence of site-specific variation and is calculated by randomly splitting in half the images from a single site, and comparing the distribution of feature values in each half with the Wilcoxon rank-sum test.…”

Novel Quantitative Imaging for Predicting Response to Therapy: Techniques and Clinical Applications

“…Various measures have been used to assess feature stability, including relative standard deviation, 26 intraclass correlation coefficient, 106 coefficient of variability, 107 and latent and preparation-induced instability score. 56 The availability of the National Cancer Institute–sponsored RIDER-CT data set, which comprises CT scans of patients with NSCLC taken 15 minutes apart, has also allowed for identification of those features that do not dramatically change between the test-retest scans.…”