<b>ThresholdROC</b>: Optimum Threshold Estimation Tools for Continuous Diagnostic Tests in <i>R</i>

Perez-Jaume, Sara; Skaltsa, Konstantina; Pallarès, Natàlia

doi:10.18637/jss.v082.i04

“…The aim in developing cutpointr was to make improved methods to estimate and validate optimal cutpoints available. While there are already several packages that aid the calculation of a wide range of metrics (López-Ratón et al 2014Perez-Jaume et al 2017), some of these rely on suboptimal procedures to estimate a cutpoint. Most use the empirical method, which is prone to high variability an to overestimating the diagnostic utility of optimal cutpoints.…”

Section: Discussionmentioning

confidence: 99%

cutpointr: Improved Estimation and Validation of Optimal Cutpoints in R

Thiele¹,

Hirschfeld²

2021

View full text Add to dashboard Cite

Optimal cutpoints" for binary classification tasks are often established by testing which cutpoint yields the best discrimination, for example the Youden index, in a specific sample. This results in "optimal" cutpoints that are highly variable and systematically overestimate the out-of-sample performance. To address these concerns, the cutpointr package offers robust methods for estimating optimal cutpoints and the out-of-sample performance. The robust methods include bootstrapping and smoothing based on kernel estimation, generalized additive models, smoothing splines, and local regression. These methods can be applied to a wide range of binary-classification and cost-based metrics. cutpointr also provides mechanisms to utilize user-defined metrics and estimation methods. The package has capabilities for parallelization of the bootstrapping, including reproducible random number generation. Furthermore, it is pipe-friendly, for example for compatibility with functions from tidyverse. Various functions for plotting receiver operating characteristic curves, precision recall graphs, bootstrap results and other representations of the data are included. The package contains example data from a study on psychological characteristics and suicide attempts suitable for applying binary classification algorithms.

show abstract

“…Linear mixed models were used to assess the association between ASP activity and ammonium, albumin, and triglycerides to account for the correlation between the measurements from the same subjects. 26 We used the Youden index maximization method, 27,28 to estimate cutoffs for ammonium, triglycerides, and albumin that optimally discriminated between patients with low ASP activity (<100 IU/L) and therapeutic ASP activity (≥100 IU/L). The corresponding 95% confidence intervals were calculated using bootstrap.…”

Section: Methodsmentioning

confidence: 99%

Lower incidence of clinical allergy with PEG‐asparaginase upfront versus the sequential use of native E. coli asparaginase followed by PEG‐ASP in pediatric patients with acute lymphoblastic leukemia

Mesegué

¹

,

Alonso‐Saladrigues

²

,

Perez-Jaume

³

et al. 2021

Hematological Oncology

View full text Add to dashboard Cite

Asparaginase (ASP) is an essential component for the acute lymphoblastic leukemia (ALL) treatment, but toxicities, such as allergy, frequently limit its use. Although the potentially lower PEG-ASP formulation immunogenicity, few studies with conflicting results have compared the allergy incidence between Escherichia coli-ASP and PEG-ASP in the same protocol. We aimed at comparing the allergy incidence in children receiving native E. coli-ASP versus PEG-ASP within the same clinical protocol (Spanish Society of Pediatric Hematology and Oncology ALL-SEHOP-PETHEMA 2013). One hundred and twenty-six children (1-19 years) diagnosed with ALL from 2013 to 2020 were included. Patients in group 1 received a sequential scheme of native E. coli-ASP 10,000 IU/m 2 intramuscularly (IM) followed by PEG-ASP 1000 IU/m 2 IM. Patients in group 2 received PEG-ASP 1000 IU/m 2 IM upfront. Clinical allergy incidence was compared between both groups. Serum ASP activity (SAA) was measured in a subgroup of patients, and silent inactivation was recorded. The cumulative incidence of clinical allergy was significantly higher in group 1 (native followed by PEG-ASP) than in group 2 (PEG-ASP upfront), 24.7% versus 4.1% (p = 0.0085). Adequate ASP activity was achieved with PEG-ASP 1000 IU/m 2 dose in most patients (median SAA 412.5 and 453.0 IU/L at days 7 and 14).

show abstract

“…The aim in developing cutpointr was to make improved methods to estimate and validate optimal cutpoints available. While there are already several packages that aid the calculation of a wide range of metrics (López-Ratón et al 2014Perez-Jaume et al 2017), some of these rely on suboptimal procedures to estimate a cutpoint. Most use the empirical method, q q q q q q q q q q q q Cutpoint Estimation ROC curve calculation 1e+02 1e+03 1e+04 1e+05 1e+06 1e+07 1e+02 1e+03 1e+04 1e+05 1e+06 1e+07 which is prone to high variability an to overestimating the diagnostic utility of optimal cutpoints.…”

Section: Discussionmentioning

confidence: 99%

cutpointr: Improved Estimation and Validation of Optimal Cutpoints in R

Thiele¹,

Hirschfeld²

2020

Preprint

View full text Add to dashboard Cite

Optimal cutpoints" for binary classification tasks are often established by testing which cutpoint yields the best discrimination, for example the Youden index, in a specific sample. This results in "optimal" cutpoints that are highly variable and systematically overestimate the out-of-sample performance. To address these concerns, the cutpointr package offers robust methods for estimating optimal cutpoints and the out-of-sample performance. The robust methods include bootstrapping and smoothing based on kernel estimation, generalized additive models, smoothing splines, and local regression. These methods can be applied to a wide range of binary-classification and cost-based metrics. cutpointr also provides mechanisms to utilize user-defined metrics and estimation methods. The package has capabilities for parallelization of the bootstrapping, including reproducible random number generation. Furthermore, it is pipe-friendly, for example for compatibility with functions from tidyverse. Various functions for plotting receiver operating characteristic curves, precision recall graphs, bootstrap results and other representations of the data are included. The package contains example data from a study on psychological characteristics and suicide attempts suitable for applying binary classification algorithms.

show abstract

ThresholdROC: Optimum Threshold Estimation Tools for Continuous Diagnostic Tests in R

Cited by 23 publications

References 16 publications

cutpointr: Improved Estimation and Validation of Optimal Cutpoints in R

cutpointr: Improved Estimation and Validation of Optimal Cutpoints in R

Lower incidence of clinical allergy with PEG‐asparaginase upfront versus the sequential use of native E. coli asparaginase followed by PEG‐ASP in pediatric patients with acute lymphoblastic leukemia

cutpointr: Improved Estimation and Validation of Optimal Cutpoints in R

Contact Info

Product

Resources

About