Multiple Instance Learning with Mixed Supervision in Gleason Grading

Bian, Hao; Shao, Zhuchen; Chen, Yang; Wang, Yifeng; Wang, Haoqian; Zhang, Jian; Zhang, Yongbing

doi:10.1007/978-3-031-16452-1_20

Cited by 6 publications

(1 citation statement)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A WSI bag will be further split into several sub-WSI bags. Inspired by the superpixel sampling strategy (Bian et al 2022), we sample at the window level to maintain 2D spatial information in each local window. Specifically, we perform m random window sampling for the rearranged feature sequence.…”

Section: Feature Generationmentioning

confidence: 99%

HVTSurv: Hierarchical Vision Transformer for Patient-Level Survival Prediction from Whole Slide Image

Shao,

Chen,

Bian

et al. 2023

AAAI

View full text Add to dashboard Cite

Survival prediction based on whole slide images (WSIs) is a challenging task for patient-level multiple instance learning (MIL). Due to the vast amount of data for a patient (one or multiple gigapixels WSIs) and the irregularly shaped property of WSI, it is difficult to fully explore spatial, contextual, and hierarchical interaction in the patient-level bag. Many studies adopt random sampling pre-processing strategy and WSI-level aggregation models, which inevitably lose critical prognostic information in the patient-level bag. In this work, we propose a hierarchical vision Transformer framework named HVTSurv, which can encode the local-level relative spatial information, strengthen WSI-level context-aware communication, and establish patient-level hierarchical interaction. Firstly, we design a feature pre-processing strategy, including feature rearrangement and random window masking. Then, we devise three layers to progressively obtain patient-level representation, including a local-level interaction layer adopting Manhattan distance, a WSI-level interaction layer employing spatial shuffle, and a patient-level interaction layer using attention pooling. Moreover, the design of hierarchical network helps the model become more computationally efficient. Finally, we validate HVTSurv with 3,104 patients and 3,752 WSIs across 6 cancer types from The Cancer Genome Atlas (TCGA). The average C-Index is 2.50-11.30% higher than all the prior weakly supervised methods over 6 TCGA datasets. Ablation study and attention visualization further verify the superiority of the proposed HVTSurv. Implementation is available at: https://github.com/szc19990412/HVTSurv.

show abstract

Section: Feature Generationmentioning

confidence: 99%