Assessing diagnostic tests by a strictly proper scoring rule

Línnet, Kristían

doi:10.1002/sim.4780080510

Cited by 15 publications

(8 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A somewhat uncritical application of the additive bootstrap approach can be found in a paper by Linnet,24 where bootstrap resampling is used to obtain a bias-corrected estimate of a score for the assessment of diagnostic tests. In this paper we have shown that a simple correction using the additive bootstrap approach is no guarantee to get unbiased estimates.…”

Section: Discussionmentioning

confidence: 99%

Resampling and cross-validation techniques: a tool to reduce bias caused by model building?

1997

View full text Add to dashboard Cite

The process of model building involved in the analysis of many medical studies may lead to a considerable amount of over-optimism with respect to the predictive ability of the 'final' regression model. In this paper we illustrate this phenomenon in a simple cutpoint model and explore to what extent bias can be reduced by using cross-validation and bootstrap resampling. These computer intensive methods are compared to an ad hoc approach and to a heuristic method. Besides illustrating all proposals with the data from a breast cancer study we perform a simulation study in order to assess the quality of the methods.

show abstract

Section: Discussionmentioning

confidence: 99%

Resampling and cross-validation techniques: a tool to reduce bias caused by model building?

1997

View full text Add to dashboard Cite

show abstract

“…Validation to quantify the prediction utility of the models was done through the bootstrapping procedure (Efron 1983(Efron 1986Efron and Gong 1983;Efron and Tibshirani 1993;Linnet 1989). This procedure (with 200 re-samples) is used to estimate the optimism in how well predicted survival probability estimates from the developed Cox model track the corresponding Kaplan-Meier survival probability (Kaplan and Meier 1958).…”

Section: Methodsmentioning

confidence: 99%

Breastfeeding as a Time-Varying–Time-Dependent Factor for Birth Spacing: Multivariate Models with Validations and Predictions

Singh

Tripathi²,

Singh³

et al. 2012

whp

View full text Add to dashboard Cite

Data used in the present study are from the National Family Health Survey (NFHS-I) 1992-93 (International Institute for Population Sciences 1995), India. Our study has developed Cox model analyses to see the effect of breastfeeding as a time-varying and time-dependent factor on birth spacing. While it is acknowledged that breastfeeding has a protective effect on birth spacing, such analysis of breastfeeding allows for a more nuanced understanding of that effect. Multivariate analysis revealed that breastfeeding, ever experience of fetal loss, education of women, employment status of women, education of husband, media exposure, survival status of index child and place of residence played an important part in extending birth space in at least one of the birth-spacing intervals (first to fifth). However, the variables varied from the first birth spacing to the fifth birth spacing. Breastfeeding is the only covariate found to be a significant protective factor associated with each birth spacing. Furthermore, this study validates the developed models with their prediction utilities for birth spacing.

show abstract

“…Full details of these statistics and their rationale have been presented in a number of publications by the ICNARC investigators, and only a brief outline is provided here. These statistics are advocated as they are "strictly proper" scoring rules, providing a quantitative assessment of the quality of fit for models with binary outcomes 31 . Such assessment is not a feature of error-rate methods or the sensitivity/ specificity methods that are derived from them (e.g.…”

Section: Methodsmentioning

confidence: 99%

Predictive Performance of Acute Physiological and Chronic Health Evaluation Releases II to IV: A Single New Zealand Centre Experience

Mann

Marshall

Woodford

et al. 2012

Anaesth Intensive Care

View full text Add to dashboard Cite

There is debate in Australia and New Zealand around the appropriate use of illness severity scoring systems in Australasian intensive care units. The international benchmark is the Acute Physiological and Chronic Health Evaluation (APACHE) system. In order to compare the performance of recent APACHE releases, we audited 2080 sequential patients admitted between 1 January 2006 and 31 March 2008 to the Middlemore Hospital intensive care unit in Auckland, New Zealand. We compared the predictive performance of the proprietary APACHE II, IIIh, IIIj and IV releases, and the performance of a 'localised' version of APACHE II containing re-estimated coefficients derived from a legacy dataset (7703 sequential patients admitted between 1 January 1997 and 31 December 2005). Discrimination assessed by receiver operating characteristic curves was highest with the APACHE III and IV releases, and significantly better than the APACHE II releases. Calibration assessed by the Hosmer-Lemeshow statistic was poor with all releases, although it was best with APACHE IV and 'localised' version of the APACHE II release. Overall accuracy assessed by the Brier Mean Probability score and Shapiro's R statistic was best with APACHE IV. Our study suggests the possibility of improved prediction in moving to APACHE IV from older releases, although broader multicentre study within the Australian and New Zealand critical care community is warranted. Our study also suggests that localisation of the APACHE system offers further opportunity to improve prediction, although these improvements may not be major without ground-up development of a new risk prediction model within our local critical care setting.

show abstract

Assessing diagnostic tests by a strictly proper scoring rule

Cited by 15 publications

References 23 publications

Resampling and cross-validation techniques: a tool to reduce bias caused by model building?

Resampling and cross-validation techniques: a tool to reduce bias caused by model building?

Breastfeeding as a Time-Varying–Time-Dependent Factor for Birth Spacing: Multivariate Models with Validations and Predictions

Predictive Performance of Acute Physiological and Chronic Health Evaluation Releases II to IV: A Single New Zealand Centre Experience

Contact Info

Product

Resources

About