Multi-scale fully convolutional neural networks for histopathology image segmentation: From nuclear aberrations to the global tissue architecture

Schmitz, Rüdiger; Madesta, Frederic; Nielsen, Maximilian; Krause, Jenny; Steurer, Stefan; Werner, René; Rösch, Thomas

doi:10.1016/j.media.2021.101996

Cited by 76 publications

(43 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, we want to stress that, even at this stage, clinicians can inject significant knowledge and provide input regarding the clinical question at hand and how they approach corresponding scenarios. In fact, designing machine learning models as a close mimicry of how humans approach analogous tasks can improve performance [15]. Therefore, the endoscopy community should definitely attempt to get involved in the development and testing phases on a continuous basis and at an early stage.…”

Section: How To Develop and Test Artificial Intelligence Systems -Clinical Contributions To Preclinical Aimentioning

confidence: 99%

Artificial intelligence in GI endoscopy: stumbling blocks, gold standards and the role of endoscopy societies

Schmitz

Werner

Repici

et al. 2021

Gut

Self Cite

View full text Add to dashboard Cite

Recent developments in artificial intelligence and machine learning have led to novel and promising applications in gastrointestinal endoscopy and beyond. Endoscopic AI has already become a topic of intensive research and marketing, and a recent review in this journal is a timely and thorough guide that defines terminology and outlines best practices for the development and assessment of AI systems. Our commentary highlights selected aspects of AI research and elaborates upon potential roles that the GI scientific community and professional societies may play in ensuring that the preclinical achievements translate into clinical reality. We believe that: Databases of diagnostic AI software need to be made transparent as should the conditions of the testing scenarios. Clinical expertise is required from the early phases of system development and testing. Medicolegal implications need to be clarified as AI gets increasingly involved in medical decision making. Finally, through the establishment of a continuously updated and curated, dedicated database for AI system evaluation ("rolling gold standard"), the GI societies could actively advance and frame this everchanging field.

show abstract

Section: How To Develop and Test Artificial Intelligence Systems -Clinical Contributions To Preclinical Aimentioning

confidence: 99%

Artificial intelligence in GI endoscopy: stumbling blocks, gold standards and the role of endoscopy societies

Schmitz

Werner

Repici

et al. 2021

Gut

Self Cite

View full text Add to dashboard Cite

show abstract

“…Most early methods for cancer segmentation are empowered by patch-based classifier with a sliding-window post processing. As FCN becomes the state-of-the-art in image segmentation, many work adopt the encoder-decoder architecture for endto-end cancer segmentation [8], [29], [30]. Among them, an ensemble model achieves the best published result on hepatocellular carcinoma segmentation [30].…”

Section: Related Workmentioning

confidence: 99%

“…Existing methods try to learn more robust features to solve the issue [7]. In [8], multiscale features have been demonstrated to be potential in cancer segmentation. However, as it can be seen from their experimental results, the model performs unsatisfactorily and only secure the 7 th place in a challenge on hepatocellular carcinoma segmentation.…”

Section: Introductionmentioning

confidence: 99%

Structure-aware scale-adaptive networks for cancer segmentation in whole-slide images

Sun,

Lopez,

Wang

et al. 2021

Preprint

View full text Add to dashboard Cite

Cancer segmentation in whole-slide images is a fundamental step for viable tumour burden estimation, which is of great value for cancer assessment. However, factors like vague boundaries or small regions dissociated from viable tumour areas make it a challenging task. Considering the usefulness of multi-scale features in various vision-related tasks, we present a structure-aware scale-adaptive feature selection method for efficient and accurate cancer segmentation. Based on a segmentation network with a popular encoder-decoder architecture, a scaleadaptive module is proposed for selecting more robust features to represent the vague, non-rigid boundaries. Furthermore, a structural similarity metric is proposed for better tissue structure awareness to deal with small region segmentation. In addition, advanced designs including several attention mechanisms and the selective-kernel convolutions are applied to the baseline network for comparative study purposes. Extensive experimental results show that the proposed structure-aware scale-adaptive networks achieve outstanding performance on liver cancer segmentation when compared to top ten submitted results in the challenge of PAIP 2019. Further evaluation on colorectal cancer segmentation shows that the scale-adaptive module improves the baseline network or outperforms the other excellent designs of attention mechanisms when considering the tradeoff between efficiency and accuracy. The sourcecode is publicly available at: https://github.com/sigma10010/histo img seg.

show abstract

“…The multi-scale structure of the images allows pathologists to analyze the image from the lowest to the highest magnification level. Pathologists analyze the images by first identifying a few regions of interest and zooming afterwards through them to visualize different details of the tissue (Schmitz et al, 2019). Each magnification level includes different types of information (Molin et al, 2016), since tissue structures appear in different ways according to their magnification level.…”

Section: Introductionmentioning

confidence: 99%

“…The characteristics of the WSIs can lead to modification of CNNs in terms of architecture, both for classification (Jimenezdel Toro et al, 2017;Lai and Deng, 2017;Gecer et al, 2018;Yang et al, 2019;Hashimoto et al, 2020) and segmentation (Ronneberger et al, 2015;Li et al, 2017;Salvi and Molinari, 2018;Schmitz et al, 2019;van Rijthoven et al, 2020), such as multi-brach networks (Yang et al, 2019;Hashimoto et al, 2020;Jain and Massoud, 2020), multiple receptive fields convolutional neural networks (Han et al, 2017;Lai and Deng, 2017;Ullah, 2017;Li et al, 2019;Zhang et al, 2020) and U-Net based networks (Bozkurt et al, 2018;van Rijthoven et al, 2020). The modification of architectures to include multiple scales is prevalent in medical imaging, since it can allow to identify examples of architecture's modifications also from other modalities, such as MRI imaging (Zeng et al, 2021a) and Gold immunochromatographic strip (GIGS) images (Zeng et al, 2019;Zeng et al, 2021b).…”

Section: Introductionmentioning

confidence: 99%

Multi_Scale_Tools: A Python Library to Exploit Multi-Scale Whole Slide Images

Marini

Otálora

Podareanu

et al. 2021

Front. Comput. Sci.

View full text Add to dashboard Cite

Algorithms proposed in computational pathology can allow to automatically analyze digitized tissue samples of histopathological images to help diagnosing diseases. Tissue samples are scanned at a high-resolution and usually saved as images with several magnification levels, namely whole slide images (WSIs). Convolutional neural networks (CNNs) represent the state-of-the-art computer vision methods targeting the analysis of histopathology images, aiming for detection, classification and segmentation. However, the development of CNNs that work with multi-scale images such as WSIs is still an open challenge. The image characteristics and the CNN properties impose architecture designs that are not trivial. Therefore, single scale CNN architectures are still often used. This paper presents Multi_Scale_Tools, a library aiming to facilitate exploiting the multi-scale structure of WSIs. Multi_Scale_Tools currently include four components: a pre-processing component, a scale detector, a multi-scale CNN for classification and a multi-scale CNN for segmentation of the images. The pre-processing component includes methods to extract patches at several magnification levels. The scale detector allows to identify the magnification level of images that do not contain this information, such as images from the scientific literature. The multi-scale CNNs are trained combining features and predictions that originate from different magnification levels. The components are developed using private datasets, including colon and breast cancer tissue samples. They are tested on private and public external data sources, such as The Cancer Genome Atlas (TCGA). The results of the library demonstrate its effectiveness and applicability. The scale detector accurately predicts multiple levels of image magnification and generalizes well to independent external data. The multi-scale CNNs outperform the single-magnification CNN for both classification and segmentation tasks. The code is developed in Python and it will be made publicly available upon publication. It aims to be easy to use and easy to be improved with additional functions.

show abstract

Multi-scale fully convolutional neural networks for histopathology image segmentation: From nuclear aberrations to the global tissue architecture

Cited by 76 publications

References 28 publications

Artificial intelligence in GI endoscopy: stumbling blocks, gold standards and the role of endoscopy societies

Artificial intelligence in GI endoscopy: stumbling blocks, gold standards and the role of endoscopy societies

Structure-aware scale-adaptive networks for cancer segmentation in whole-slide images

Multi_Scale_Tools: A Python Library to Exploit Multi-Scale Whole Slide Images

Contact Info

Product

Resources

About