StoHisNet: A hybrid multi-classification model with CNN and Transformer for gastric pathology images

Fu, Bangkang; Zhang, Mudan; He, Junjie; Cao, Ying; Guo, Yuchen; Wang, Rongpin

doi:10.1016/j.cmpb.2022.106924

Cited by 42 publications

(10 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The past two years have seen a surge in popularity of transformer modeling for common computational pathology tasks such as WSI segmentation [40,43,44] and histology image classification [41,[45][46][47]. Transformers have also been used for pathologist-level question-answering from histological imaging [39], predicting pathologists' visual attention [48], and for pathology text mining [49].…”

Section: Introductionmentioning

confidence: 99%

“…Transformers have also been used for pathologist-level question-answering from histological imaging [39], predicting pathologists' visual attention [48], and for pathology text mining [49]. Nearly all applications of transformer-based approaches to whole slide imaging implement vision transformers (ViT), including recent works that combine CNNs and transformers [45]. In contrast, our multi-stage pipeline extracts expert-defined features from segmented structures before deriving WSI context using transformer encoders.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Spatially Aware Transformer Networks for Contextual Prediction of Diabetic Nephropathy Progression from Whole Slide Images

Shickel

Lucarelli

Rao

et al. 2023

Preprint

View full text Add to dashboard Cite

Diabetic nephropathy (DN) in the context of type 2 diabetes is the leading cause of end-stage renal disease (ESRD) in the United States. DN is graded based on glomerular morphology and has a spatially heterogeneous presentation in kidney biopsies that complicates pathologists predictions of disease progression. Artificial intelligence and deep learning methods for pathology have shown promise for quantitative pathological evaluation and clinical trajectory estimation; but, they often fail to capture large-scale spatial anatomy and relationships found in whole slide images (WSIs). In this study, we present a transformer-based, multi-stage ESRD prediction framework built upon nonlinear dimensionality reduction, relative Euclidean pixel distance embeddings between every pair of observable glomeruli, and a corresponding spatial self-attention mechanism for a robust contextual representation. We developed a deep transformer network for encoding WSI and predicting future ESRD using a dataset of 56 kidney biopsy WSIs from DN patients at Seoul National University Hospital. Using a leave-one-out cross-validation scheme, our modified transformer framework outperformed RNNs, XGBoost, and logistic regression baseline models, and resulted in an area under the receiver operating characteristic curve (AUC) of 0.97 (95% CI: 0.90-1.00) for predicting two-year ESRD, compared with an AUC of 0.86 (95% CI: 0.66-0.99) without our relative distance embedding, and an AUC of 0.76 (95% CI: 0.59-0.92) without a denoising autoencoder module. While the variability and generalizability induced by smaller sample sizes are challenging, our distance-based embedding approach and overfitting mitigation techniques yielded results that suggest opportunities for future spatially aware WSI research using limited pathology datasets.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Spatially Aware Transformer Networks for Contextual Prediction of Diabetic Nephropathy Progression from Whole Slide Images

Shickel

Lucarelli

Rao

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Before this paper, research on HI classification based on deep learning has utilized various network structures, such as modified convolutional neural networks (CNN) (Hameed et al 2020, Wu et al 2020, Aatresh et al 2021, Elmannai et al 2021, Chattopadhyay et al 2022, Transformer networks (Tummala et al 2022), and hybrid architectures (Zou et al 2021, Fu et al 2022. However, we observed that all these methods to some extent overlook certain information present in HI.…”

Section: Introductionmentioning

confidence: 99%

“…CNNs tend to pay less attention to global information, while Transformer networks struggle to capture local information. Combining the two types of neural networks may resolve this problem, but previous works such as (Zou et al 2021, Fu et al 2022 only combined the highlevel features of the two networks while neglecting some low-level information communication. Additionally, previous works have primarily focused on evolving network structures while disregarding the characteristics of HIs.…”

Section: Introductionmentioning

confidence: 99%

Dual-branch hybrid encoding embedded network for histopathology image classification

Li,

Hu,

Qiu

et al. 2023

Phys. Med. Biol.

View full text Add to dashboard Cite

Objective. Learning-based histopathology image (HI) classification methods serve as important tools for auxiliary diagnosis in the prognosis stage. However, most existing methods are focus on a single target cancer due to inter-domain differences among different cancer types, limiting their applicability to different cancer types. To overcome these limitations, this paper presents a high-performance HI classification method that aims to address inter-domain differences and provide an improved solution for reliable and practical HI classification. Approach. Firstly, we collect a high-quality hepatocellular carcinoma (HCC) dataset with enough data to verify the stability and practicability of the method. Secondly, a novel dual-branch hybrid encoding embedded network is proposed, which integrates the feature extraction capabilities of convolutional neural network and Transformer. This well-designed structure enables the network to extract diverse features while minimizing redundancy from a single complex network. Lastly, we develop a salient area constraint loss function tailored to the unique characteristics of HIs to address inter-domain differences and enhance the robustness and universality of the methods. Main results. Extensive experiments have conducted on the proposed HCC dataset and two other publicly available datasets. The proposed method demonstrates outstanding performance with an impressive accuracy of 99.09% on the HCC dataset and achieves state-of-the-art results on the other two public datasets. These remarkable outcomes underscore the superior performance and versatility of our approach in multiple HI classification. Significance. The advancements presented in this study contribute to the field of HI analysis by providing a reliable and practical solution for multiple cancer classification, potentially improving diagnostic accuracy and patient outcomes. Our code is available at https://github.com/lms-design/DHEE-net.

show abstract

“…), Sub-database C (80 × 80 pixels). GasHisSDB provides the ability to distinguish between classical machine learning classifier performance and deep learning classifier performance ( 13 ). Details are given in Section 2.1.…”

Section: Introductionmentioning

confidence: 99%

A comparative study of gastric histopathology sub-size image classification: From linear regression to visual transformer

Chen

Liu

et al. 2022

Front. Med.

View full text Add to dashboard Cite

IntroductionGastric cancer is the fifth most common cancer in the world. At the same time, it is also the fourth most deadly cancer. Early detection of cancer exists as a guide for the treatment of gastric cancer. Nowadays, computer technology has advanced rapidly to assist physicians in the diagnosis of pathological pictures of gastric cancer. Ensemble learning is a way to improve the accuracy of algorithms, and finding multiple learning models with complementarity types is the basis of ensemble learning. Therefore, this paper compares the performance of multiple algorithms in anticipation of applying ensemble learning to a practical gastric cancer classification problem.MethodsThe complementarity of sub-size pathology image classifiers when machine performance is insufficient is explored in this experimental platform. We choose seven classical machine learning classifiers and four deep learning classifiers for classification experiments on the GasHisSDB database. Among them, classical machine learning algorithms extract five different image virtual features to match multiple classifier algorithms. For deep learning, we choose three convolutional neural network classifiers. In addition, we also choose a novel Transformer-based classifier.ResultsThe experimental platform, in which a large number of classical machine learning and deep learning methods are performed, demonstrates that there are differences in the performance of different classifiers on GasHisSDB. Classical machine learning models exist for classifiers that classify Abnormal categories very well, while classifiers that excel in classifying Normal categories also exist. Deep learning models also exist with multiple models that can be complementarity.DiscussionSuitable classifiers are selected for ensemble learning, when machine performance is insufficient. This experimental platform demonstrates that multiple classifiers are indeed complementarity and can improve the efficiency of ensemble learning. This can better assist doctors in diagnosis, improve the detection of gastric cancer, and increase the cure rate.

show abstract

StoHisNet: A hybrid multi-classification model with CNN and Transformer for gastric pathology images

Cited by 42 publications

References 24 publications

Spatially Aware Transformer Networks for Contextual Prediction of Diabetic Nephropathy Progression from Whole Slide Images

Spatially Aware Transformer Networks for Contextual Prediction of Diabetic Nephropathy Progression from Whole Slide Images

Dual-branch hybrid encoding embedded network for histopathology image classification

A comparative study of gastric histopathology sub-size image classification: From linear regression to visual transformer

Contact Info

Product

Resources

About