CNN cascades for segmenting sparse objects in gigapixel whole slide images

Gadermayr, Michael; Dombrowski, Ann-Kathrin; Klinkhammer, Barbara M.; Boor, Peter; Merhof, Dorit

doi:10.1016/j.compmedimag.2018.11.002

Cited by 73 publications

(43 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The number of studies aiming to develop deep learning applications for nephropathology has increased rapidly over the past years. 11,14,15 Pretrained neural networks have successfully been applied for the distinction between glomerular and nonglomerular regions. Pedraza et al 14 trained a CNN for the detection of glomeruli in preselected areas on PAS-stained human renal biopsies.…”

Section: Discussionmentioning

confidence: 99%

“…Gadermayr et al 11 investigated two cascades where two U-nets are combined with a sliding window CNN: 80% of glomerular objects where found with a DC of 0.90 or higher when detection and segmentation were combined. 11 As a limitation, this work focused only on segmentation of glomeruli, whereas the simultaneous segmentation and classification of the whole renal cortex can be of great added value in kidney diagnostics and research. Our first objective was therefore to train a CNN for the segmentation of PAS-stained kidney sections into ten significant tissue classes.…”

Section: Discussionmentioning

confidence: 99%

“…10 Advances in machine learning (mainly the emergence of deep neural networks, collectively called "deep learning") combined with the possibility to digitize entire tissue sections at microscopic resolution within minutes (whole-slide images [WSIs]) have paved the way for more advanced digital analysis of histopathologic images. [11][12][13] Deep learning techniques allow autonomous learning of increasingly complex structures during the transformation from input (WSI) to desired output (e.g., structure detection). The most widely applied deep learning models for analysis of images are so-called convolutional neural networks (CNNs).…”

mentioning

confidence: 99%

See 2 more Smart Citations

Deep Learning–Based Histopathologic Assessment of Kidney Tissue

Hermsen¹,

Bel²,

Boer³

et al. 2019

JASN

279

204

View full text Add to dashboard Cite

BackgroundThe development of deep neural networks is facilitating more advanced digital analysis of histopathologic images. We trained a convolutional neural network for multiclass segmentation of digitized kidney tissue sections stained with periodic acid–Schiff (PAS).MethodsWe trained the network using multiclass annotations from 40 whole-slide images of stained kidney transplant biopsies and applied it to four independent data sets. We assessed multiclass segmentation performance by calculating Dice coefficients for ten tissue classes on ten transplant biopsies from the Radboud University Medical Center in Nijmegen, The Netherlands, and on ten transplant biopsies from an external center for validation. We also fully segmented 15 nephrectomy samples and calculated the network’s glomerular detection rates and compared network-based measures with visually scored histologic components (Banff classification) in 82 kidney transplant biopsies.ResultsThe weighted mean Dice coefficients of all classes were 0.80 and 0.84 in ten kidney transplant biopsies from the Radboud center and the external center, respectively. The best segmented class was “glomeruli” in both data sets (Dice coefficients, 0.95 and 0.94, respectively), followed by “tubuli combined” and “interstitium.” The network detected 92.7% of all glomeruli in nephrectomy samples, with 10.4% false positives. In whole transplant biopsies, the mean intraclass correlation coefficient for glomerular counting performed by pathologists versus the network was 0.94. We found significant correlations between visually scored histologic components and network-based measures.ConclusionsThis study presents the first convolutional neural network for multiclass segmentation of PAS-stained nephrectomy samples and transplant biopsies. Our network may have utility for quantitative studies involving kidney histopathology across centers and provide opportunities for deep learning applications in routine diagnostics.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

mentioning

confidence: 99%

See 1 more Smart Citation

Deep Learning–Based Histopathologic Assessment of Kidney Tissue

Hermsen¹,

Bel²,

Boer³

et al. 2019

JASN

279

204

View full text Add to dashboard Cite

show abstract

“…While some studies have validated DL models analyzing the structures other than the glomeruli, such as the tubules, blood vessels, and interstitium [7][8][9][10], many studies have focused on the glomeruli, which present various histological findings essential for diagnosis. As a first step in the automation of this diagnostic procedure, detection of a glomerulus in a whole slide image (WSI) of renal tissue specimens has been recently attempted in many studies with the use of methods to define various features [11][12][13][14][15][16][17][18][19][20][21][22][23][24] or using convolutional neural networks (CNNs) [25], such as InceptionV3 [26], AlexNet [27], U-Net [28], R-CNN [29,30], or DeepLab V2 ResNet [31].…”

Section: Introductionmentioning

confidence: 99%

Classification of glomerular pathological findings using deep learning and nephrologist–AI collective intelligence approach

Uchino

Suzuki

Sato

et al. 2020

Preprint

View full text Add to dashboard Cite

Background: Automated classification of glomerular pathological findings is potentially beneficial in establishing an efficient and objective diagnosis in renal pathology. While previous studies have verified the artificial intelligence (AI) models for the classification of global sclerosis and glomerular cell proliferation, there are several other glomerular pathological findings required for diagnosis, and the comprehensive models for the classification of these major findings have not yet been reported. Whether the cooperation between these AI models and clinicians improves diagnostic performance also remains unknown. Here, we developed AI models to classify glomerular images for major findings required for pathological diagnosis and investigated whether those models could improve the diagnostic performance of nephrologists. Methods: We used a dataset of 283 kidney biopsy cases comprising 15888 glomerular images that were annotated by a total of 25 nephrologists. AI models to classify seven pathological findings: global sclerosis, segmental sclerosis, endocapillary proliferation, mesangial matrix accumulation, mesangial cell proliferation, crescent, and basement membrane structural changes, were constructed using deep learning by fine-tuning of InceptionV3 convolutional neural network. Subsequently, we compared the agreement to truth labels between majority decision among nephrologists with or without the AI model as a voter. Results: Our model for global sclerosis showed high performance (area under the curve: periodic acid-Schiff, 0.986; periodic acid methenamine silver, 0.983); the models for the other findings also showed performance close to those of nephrologists. By adding the AI model output to majority decision among nephrologists, the sensitivity and specificity were significantly improved in 9 of 14 constructed models compared to those of nephrologists alone. Conclusion: Our study showed a proof-of-concept for the classification of multiple glomerular findings in a comprehensive method of deep learning and suggested its potential effectiveness in improving diagnostic accuracy of clinicians.

show abstract

“…The researchers adopted Deep Learning technology to achieve different goals: some focused on the automated identification of the main microscopic structures (glomeruli, tubules, vessels etc.) [21][22][23][24][25] , some authors worked on the segmentation of the identified structures 2 , and others have used Convolutional Neural Networks to obtain automated clinical classifications 1,2 . We can consider our approach as part of the strategies for developing a tool for assisting in automated clinical classification.…”

Section: Discussionmentioning

confidence: 99%

Evaluation of the Classification Accuracy of the Kidney Biopsy Direct Immunofluorescence through Convolutional Neural Networks

Ligabue

Pollastri

Fontana

et al. 2020

CJASN

View full text Add to dashboard Cite

Background and objectivesImmunohistopathology is an essential technique in the diagnostic workflow of a kidney biopsy. Deep learning is an effective tool in the elaboration of medical imaging. We wanted to evaluate the role of a convolutional neural network as a support tool for kidney immunofluorescence reporting.Design, setting, participants, & measurementsHigh-magnification (×400) immunofluorescence images of kidney biopsies performed from the year 2001 to 2018 were collected. The report, adopted at the Division of Nephrology of the AOU Policlinico di Modena, describes the specimen in terms of “appearance,” “distribution,” “location,” and “intensity” of the glomerular deposits identified with fluorescent antibodies against IgG, IgA, IgM, C1q and C3 complement fractions, fibrinogen, and κ- and λ-light chains. The report was used as ground truth for the training of the convolutional neural networks.ResultsIn total, 12,259 immunofluorescence images of 2542 subjects undergoing kidney biopsy were collected. The test set analysis showed accuracy values between 0.79 (“irregular capillary wall” feature) and 0.94 (“fine granular” feature). The agreement test of the results obtained by the convolutional neural networks with respect to the ground truth showed similar values to three pathologists of our center. Convolutional neural networks were 117 times faster than human evaluators in analyzing 180 test images. A web platform, where it is possible to upload digitized images of immunofluorescence specimens, is available to evaluate the potential of our approach.ConclusionsThe data showed that the accuracy of convolutional neural networks is comparable with that of pathologists experienced in the field.

show abstract

CNN cascades for segmenting sparse objects in gigapixel whole slide images

Cited by 73 publications

References 18 publications

Deep Learning–Based Histopathologic Assessment of Kidney Tissue

Deep Learning–Based Histopathologic Assessment of Kidney Tissue

Classification of glomerular pathological findings using deep learning and nephrologist–AI collective intelligence approach

Evaluation of the Classification Accuracy of the Kidney Biopsy Direct Immunofluorescence through Convolutional Neural Networks

Contact Info

Product

Resources

About