A survey of Transformer applications for histopathological image analysis: New developments and future directions

Atabansi, Chukwuemeka Clinton; Nie, Jing; Liu, Haijun; Song, Qianqian; Yan, Lingfeng; Zhou, Xichuan

doi:10.1186/s12938-023-01157-0

Cited by 16 publications

(2 citation statements)

References 114 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some researches [23,24] demonstrate that DCNNs perform better on small datasets, thanks to the inductive bias, which helps them to learn spatial relevance effectively. While other works [25][26][27] argue in favor of ViTs, showing that they are highly robust, attend to overall structural information, and are less biased towards textures. Nevertheless, both DL architectures may suffer from overfitting, poor generalization, and reproducibility issues, leading to overconfident predictions on new (external) data.…”

Section: Artifact Report Segmentation Maskmentioning

confidence: 92%

Equipping Computational Pathology Systems with Artifact Processing Pipelines: A Showcase for Computation and Performance Trade-offs

Kanwal,

Khoraminia,

Kiraz

et al. 2024

Preprint

View full text Add to dashboard Cite

Background: Histopathology is a gold standard for cancer diagnosis. It involves extracting tissue specimens from suspicious areas to prepare a glass slide for a microscopic examination. However, histological tissue processing procedures result in the introduction of artifacts, which are ultimately transferred to the digitized version of glass slides, known as whole slide images (WSIs). Artifacts are diagnostically irrelevant areas and may result in wrong predictions from deep learning (DL) algorithms. Therefore, detecting and excluding artifacts in the computational pathology (CPATH) system is essential for reliable automated diagnosis. Methods: In this paper, we propose a mixture of experts (MoE) scheme for detecting five notable artifacts, including damaged tissue, blur, folded tissue, air bubbles, and histologically irrelevant blood from WSIs. First, we train independent binary DL models as experts to capture particular artifact morphology. Then, we ensemble their predictions using a fusion mechanism. We apply probabilistic thresholding over the final probability distribution to improve the sensitivity of the MoE. We developed four DL pipelines to evaluate computational and performance trade-offs. These include two MoEs and two multiclass models of state-of-the-art deep convolutional neural networks (DCNNs) and vision transformers (ViTs). These DL pipelines are quantitatively and qualitatively evaluated on external and out-of-distribution (OoD) data to assess generalizability and robustness for artifact detection application. Results: We extensively evaluated the proposed MoE and multiclass models. DCNNs-based MoE and ViTs-based MoE schemes outperformed simpler multiclass models and were tested on datasets from different hospitals and cancer types, where MoE using (MobiletNet) DCNNs yielded the best results. The proposed MoE yields 86.15% F1 and 97.93% sensitivity scores on unseen data, retaining less computational cost for inference than MoE using ViTs. This best performance of MoEs comes with relatively higher computational trade-offs than multiclass models. Furthermore, we apply post-processing to create an artifact segmentation mask, a potential artifact-free RoI map, a quality report, and an artifact-refined WSI for further computational analysis. During the qualitative evaluation, pathologists assessed the predictive performance of MoEs over OoD WSIs. They rated artifact detection and artifact-free area preservation, where the highest agreement translated to a Cohen kappa of 0.82, indicating substantial agreement for the overall diagnostic usability of the DCNN-based MoE scheme. Conclusion: The proposed artifact detection pipeline will not only ensure reliable CPATH predictions but may also provide quality control. In this work, the best-performing pipeline for artifact detection is MoE with DCNNs. Our detailed experiments show that there is always a trade-off between performance and computational complexity, and no straightforward DL solution equally suits all types of data and applications. The code and HistoArtifacts dataset can be found online at Github and Zenodo, respectively.

show abstract

Section: Artifact Report Segmentation Maskmentioning

confidence: 92%

Equipping Computational Pathology Systems with Artifact Processing Pipelines: A Showcase for Computation and Performance Trade-offs

Kanwal,

Khoraminia,

Kiraz

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…Some researches [23, 24] demonstrate that DCNNs perform better on small datasets, thanks to the inductive bias, which helps them to learn spatial relevance effectively. While other works [25–27] argue in favor of ViTs, showing that they are highly robust, attend to overall structural information, and are less biased towards textures. Nevertheless, both DL architectures may suffer from overfitting, poor generalization, and reproducibility issues, leading to overconfident predictions on new (external) data.…”

Section: Introductionmentioning

confidence: 97%

Equipping Computational Pathology Systems with Artifact Processing Pipelines: A Showcase for Computation and Performance Trade-offs

Kanwal,

Khoraminia,

Kiraz

et al. 2024

Preprint

View full text Add to dashboard Cite

Background: Histopathology is a gold standard for cancer diagnosis. It involves extracting tissue specimens from suspicious areas to prepare a glass slide for a microscopic examination. However, histological tissue processing procedures result in the introduction of artifacts, which are ultimately transferred to the digitized version of glass slides, known as whole slide images (WSIs). Artifacts are diagnostically irrelevant areas and may result in wrong predictions from deep learning (DL) algorithms. Therefore, detecting and excluding artifacts in the computational pathology (CPATH) system is essential for reliable automated diagnosis. Methods: In this paper, we propose a mixture of experts (MoE) scheme for detecting five notable artifacts, including damaged tissue, blur, folded tissue, air bubbles, and histologically irrelevant blood from WSIs. First, we train independent binary DL models as experts to capture particular artifact morphology. Then, we ensemble their predictions using a fusion mechanism. We apply probabilistic thresholding over the final probability distribution to improve the sensitivity of the MoE. We developed four DL pipelines to evaluate computational and performance trade-offs. These include two MoEs and two multiclass models of state-of-the-art deep convolutional neural networks (DCNNs) and vision transformers (ViTs). These DL pipelines are quantitatively and qualitatively evaluated on external and out-of-distribution (OoD) data to assess generalizability and robustness for artifact detection application. Results: We extensively evaluated the proposed MoE and multiclass models. DCNNs-based MoE and ViTs-based MoE schemes outperformed simpler multiclass models and were tested on datasets from different hospitals and cancer types, where MoE using (MobiletNet) DCNNs yielded the best results. The proposed MoE yields 86.15% F1 and 97.93% sensitivity scores on unseen data, retaining less computational cost for inference than MoE using ViTs. This best performance of MoEs comes with relatively higher computational trade-offs than multiclass models. Furthermore, we apply post-processing to create an artifact segmentation mask, a potential artifact-free RoI map, a quality report, and an artifact-refined WSI for further computational analysis. During the qualitative evaluation, pathologists assessed the predictive performance of MoEs over OoD WSIs. They rated artifact detection and artifact-free area preservation, where the highest agreement translated to a Cohen kappa of 0.82, indicating substantial agreement for the overall diagnostic usability of the DCNN-based MoE scheme. Conclusion: The proposed artifact detection pipeline will not only ensure reliable CPATH predictions but may also provide quality control. In this work, the best-performing pipeline for artifact detection is MoE with DCNNs. Our detailed experiments show that there is always a trade-off between performance and computational complexity, and no straightforward DL solution equally suits all types of data and applications. The code and dataset for training and development can be found online at Github and Zenodo, respectively.

show abstract

Dense Prediction of Cell Centroids Using Tissue Context and Cell Refinement

Millward,

He,

Nibali

2024

Lecture Notes in Computer Science

View full text Add to dashboard Cite

A survey of Transformer applications for histopathological image analysis: New developments and future directions

Cited by 16 publications

References 114 publications

Equipping Computational Pathology Systems with Artifact Processing Pipelines: A Showcase for Computation and Performance Trade-offs

Equipping Computational Pathology Systems with Artifact Processing Pipelines: A Showcase for Computation and Performance Trade-offs

Equipping Computational Pathology Systems with Artifact Processing Pipelines: A Showcase for Computation and Performance Trade-offs

Dense Prediction of Cell Centroids Using Tissue Context and Cell Refinement

Contact Info

Product

Resources

About