Machine Learning for Outcome Prediction in First-Line Surgery of Prolactinomas

Abstract

BackgroundFirst-line surgery for prolactinomas has gained increasing acceptance, but the indication still remains controversial. Thus, accurate prediction of unfavorable outcomes after upfront surgery in prolactinoma patients is critical for the triage of therapy and for interdisciplinary decision-making.ObjectiveTo evaluate whether contemporary machine learning (ML) methods can facilitate this crucial prediction task in a large cohort of prolactinoma patients with first-line surgery, we investigated the performance of various classes of supervised classification algorithms. The primary endpoint was ML-applied risk prediction of long-term dopamine agonist (DA) dependency. The secondary outcome was the prediction of the early and long-term control of hyperprolactinemia.MethodsBy jointly examining two independent performance metrics – the area under the receiver operating characteristic (AUROC) and the Matthews correlation coefficient (MCC) – in combination with a stacked super learner, we present a novel perspective on how to assess and compare the discrimination capacity of a set of binary classifiers.ResultsWe demonstrate that for upfront surgery in prolactinoma patients there are not a one-algorithm-fits-all solution in outcome prediction: different algorithms perform best for different time points and different outcomes parameters. In addition, ML classifiers outperform logistic regression in both performance metrics in our cohort when predicting the primary outcome at long-term follow-up and secondary outcome at early follow-up, thus provide an added benefit in risk prediction modeling. In such a setting, the stacking framework of combining the predictions of individual base learners in a so-called super learner offers great potential: the super learner exhibits very good prediction skill for the primary outcome (AUROC: mean 0.9, 95% CI: 0.92 – 1.00; MCC: 0.85, 95% CI: 0.60 – 1.00). In contrast, predicting control of hyperprolactinemia is challenging, in particular in terms of early follow-up (AUROC: 0.69, 95% CI: 0.50 – 0.83) vs. long-term follow-up (AUROC: 0.80, 95% CI: 0.58 – 0.97). It is of clinical importance that baseline prolactin levels are by far the most important outcome predictor at early follow-up, whereas remissions at 30 days dominate the ML prediction skill for DA-dependency over the long-term.ConclusionsThis study highlights the performance benefits of combining a diverse set of classification algorithms to predict the outcome of first-line surgery in prolactinoma patients. We demonstrate the added benefit of considering two performance metrics jointly to assess the discrimination capacity of a diverse set of classifiers.

Commentary: Automated Machine Learning Model Development for Intracranial Aneurysm Treatment Outcome Prediction: A Feasibility Study

“…The authors mention further drawbacks of an AutoML approach, for example in terms of the black-box problems, which could be tackled by novel interpretation techniques such as SHAP values. However, while these techniques provide information regarding the importance of individual predictors, we argue that by considering the predictive performance of an ensemble of classifiers for two performance metrics jointly provides additional valuable information to compare different algorithms (9). Thus, an illustration of the performance of various algorithms within the search for the optimal pipeline of an AutoML application might provide additional and helpful information regarding the performance and robustness of both standard statistical methods such as multivariable logistic regression and modern machine learning methods.…”

Prolactinomas in adolescent and elderly patients—A comparative long-term analysis

“…While amenorrhea in female patients is clinically apparent and easily detected, non-specific symptoms of hypogonadism in men, such as loss of libido, are frequently not reported. Subsequently, a higher prevalence of macroprolactinomas in men than in women has been recorded ( 32 , 33 ). As macroprolactinomas are associated with longer lasting hyperprolactinemia and related hypogonadism ( 34 , 35 ), the significantly higher rates of baseline PRL levels in the EO cohort probably reflect on the longer disease duration in more oligosymptomatic men ( 36 , 37 ).…”

The diagnostic yield of inferior petrosal sinus sampling in Cushing syndrome in the era of ovine CRH shortage

“…A central-to-peripheral (C:P) gradient ≥ 2.0 before and/or ≥ 3.0 after oCRH stimulation pointed out a central ACTH source [ 22 ], and lateralization with an intersinus gradient ≥ 1.4 [ 21 ]. A transnasal transsphenoidal surgical (TTS) approach to the pituitary gland was used [ 3 , 5 – 7 , 18 , 19 ]. In case of negative intraoperative adenoma identification, the lateral third of the pituitary gland was resected on the side predicted by IPSS [ 2 ].…”