Generalizable and Automated Classification of TNM Stage from Pathology Reports with External Validation

Abstract

Cancer staging is an essential clinical attribute informing patient prognosis and clinical trial eligibility. However, it is not routinely recorded in structured electronic health records. Here, we present a generalizable method for the automated classification of TNM stage directly from pathology report text. We train a BERT-based model using publicly available pathology reports across approximately 7,000 patients and 23 cancer types. We explore the use of different model types, with differing input sizes, parameters, and model architectures. Our final model goes beyond term-extraction, inferring TNM stage from context when it is not included in the report text explicitly. As external validation, we test our model on almost 8,000 pathology reports from Columbia University Medical Center, finding that our trained model achieved an AU-ROC of 0.815-0.942. This suggests that our model can be applied broadly to other institutions without additional institution-specific fine-tuning.

Using Machine Learning Algorithms in Determining the Stage of Breast Cancer from Pathology Reports

“…Many studies have so far used machine learning algorithms for breast cancer classification and diagnosis [25]. Other studies tried to use different machine learning methods [26], combination of machine learning and rule-based approach [27], or very recently use large language models [28] to predict TNM stages of breast cancer from pathology text reports. The objective of the study was to propose a simple machine learning based model that can automatically (and with minimal data preparation) classify clinical/surgical pathology reports of breast cancer based on TNM stages.…”

Applications of Large Language Models in Pathology

“…Fijacko et al performed multinomial classification of abstract titles using the ChatGPT-4 application programming interface (API), through a python function call with predefined prompts, demonstrating the effectiveness of LLM-based approaches in bibliometric analysis [44]. Using optical character recognition to convert pathology reports into a textual format, Kefeli and Tatonetti trained several BERT-based models for TNM stage and cancer type classification [45,46]. Fang and Wang used several BERT models pre-trained on scientific literature for multi-label topic classification, achieving F1-scores over 90% [47].…”