Prediction models for clustered data: comparison of a random intercept and standard regression model

Bouwmeester, Walter; Twisk, Jos W. R.; Kappen, Teus H.; Klei, Wilton A. van; Moons, Karel G.M.; Vergouwe, Yvonne

doi:10.1186/1471-2288-13-19

Cited by 77 publications

(104 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Patients were not clustered at the level of center (a single center was used) or treating physician though there is evidence that accounting for clusters, when possible, may improve resultant predictions. [35]…”

Section: Methodsmentioning

confidence: 99%

Beyond discrimination: A comparison of calibration methods and clinical usefulness of predictive models of readmission risk

Walsh

Sharman

Hripcsak

2017

Journal of Biomedical Informatics

View full text Add to dashboard Cite

Background Prior to implementing predictive models in novel settings, analyses of calibration and clinical usefulness remain as important as discrimination, but they are not frequently discussed. Calibration is a model’s reflection of actual outcome prevalence in its predictions. Clinical usefulness refers to the utilities, costs, and harms of using a predictive model in practice. A decision analytic approach to calibrating and selecting an optimal intervention threshold may help maximize the impact of readmission risk and other preventive interventions. Objectives To select a pragmatic means of calibrating predictive models that requires a minimum amount of validation data and that performs well in practice. To evaluate the impact of miscalibration on utility and cost via clinical usefulness analyses. Materials and Methods Observational, retrospective cohort study with electronic health record data from 120,000 inpatient admissions at an urban, academic center in Manhattan. The primary outcome was thirty-day readmission for three causes: all-cause, congestive heart failure, and chronic coronary atherosclerotic disease. Predictive modeling was performed via L1-regularized logistic regression. Calibration methods were compared including Platt Scaling, Logistic Calibration, and Prevalence Adjustment. Performance of predictive modeling and calibration was assessed via discrimination (c-statistic), calibration (Spiegelhalter Z-statistic, Root Mean Square Error [RMSE] of binned predictions, Sanders and Murphy Resolutions of the Brier Score, Calibration Slope and Intercept), and clinical usefulness (utility terms represented as costs). The amount of validation data necessary to apply each calibration algorithm was also assessed. Results C-statistics by diagnosis ranged from 0.7 for all-cause readmission to 0.86 (0.78–0.93) for congestive heart failure. Logistic Calibration and Platt Scaling performed best and this difference required analyzing multiple metrics of calibration simultaneously, in particular Calibration Slopes and Intercepts. Clinical usefulness analyses provided optimal risk thresholds, which varied by reason for readmission, outcome prevalence, and calibration algorithm. Utility analyses also suggested maximum tolerable intervention costs, e.g. $1,720 for all-cause readmissions based on a published cost of readmission of $11,000. Conclusions Choice of calibration method depends on availability of validation data and on performance. Improperly calibrated models may contribute to higher costs of intervention as measured via clinical usefulness. Decision-makers must understand underlying utilities or costs inherent in the use-case at hand to assess usefulness and will obtain the optimal risk threshold to trigger intervention and intervention cost limits as a result.

show abstract

Section: Methodsmentioning

confidence: 99%

Beyond discrimination: A comparison of calibration methods and clinical usefulness of predictive models of readmission risk

Walsh

Sharman

Hripcsak

2017

Journal of Biomedical Informatics

View full text Add to dashboard Cite

show abstract

“…For example, in 2012, Anooj et al [8] develop a fuzzy rule-based decision support system for prediction of heart disease. In 2013, Bouwmeester et al [9] use the multivariate logistic regression technique for developing the risk prediction model, in which a linear combination of predictors associated with multiple symptoms and environmental data are used to fit a logarithmic transformation of the probability of the tested disease. Actually, to flourish the medical industry, more and more data analysis companies are encouraged to mine new knowledge for efficient medical service.…”

Section: Related Workmentioning

confidence: 99%

PSLP: Privacy-preserving single-layer perceptron learning for e-Healthcare

Wang

Huang

2015

2015 10th International Conference on Information, Communications and Signal Processing (ICICS)

View full text Add to dashboard Cite

In big data era, the explosive data mining techniques are used as popular tools to mine useful knowledge for the hospitals. However, considering the complexity of these techniques, the hospitals tend to outsource both data and calculations to computationally powerful cloud, which however poses a potential threat to user's privacy. In this paper, in order to address the privacy challenge, based on Paillier homomorphic cryptosystem, we propose a feasible privacy-preserving single-layer perceptron scheme, named PSLP. Specifically, in the proposed PSLP scheme, a hospital outsources the sensitive medical information to the cloud in ciphertext, and then the cloud can execute the privacypreserving neural network training to obtain the disease model. Detailed security analysis shows the proposed PSLP can really achieve privacy-preserving property. In addition, extensive performance evaluations also demonstrate it is feasible in terms of computational cost and communication overhead.

show abstract

“…For example, in 2012, Anooj et al [11] develop a fuzzy rule-based decision support system for prediction of heart disease. In 2013, Bouwmeester et al [7] use the multivariate logistic regression technique for developing the risk prediction model, in which a linear combination of predictors associated with multiple symptoms and environmental data are used to fit a logarithmic transformation of the probability of the tested disease. Actually, to flourish the medical industry, more and more data analysis companies are encouraged to mine new knowledge for efficient medical service.…”

Section: Related Workmentioning

confidence: 99%

“…Many diagnostic prediction models, which combine patient characteristics and environmental data to predict the presence or absence of a certain diagnosis, have been well developed [7]. The association between each symptom and a disease is expressed by the odds ratio (OR), which is the ratio of odds in a group of individuals having the symptom to that of those who do not have.…”

Section: A Model Of the Disease Riskmentioning

confidence: 99%

See 1 more Smart Citation

PGuide: An Efficient and Privacy-Preserving Smartphone-Based Pre-Clinical Guidance Scheme

Wang

Huang

2014

2015 IEEE Global Communications Conference (GLOBECOM)

View full text Add to dashboard Cite

With the pervasiveness of smartphones, mobile eHealthcare has attracted considerable attention in recent years. Disease risk prediction, as it can assist in predicting user's disease with big data analytics techniques, has become one of important topics in the field of e-Healthcare. However, if the privacy issue is not well addressed, disease risk predication cannot step into its flourish. Aiming at addressing this challenge, in this paper, we propose a new efficient and privacy-preserving pre-clinical guidance scheme, called PGuide, which offers selfdiagnosis service to medical users in a privacy-preserving way. In specific, to motivate medical users to provide more detailed health profile for accurate disease risk prediction, we introduce a privacy-preserving comparison protocol PPCP in the PGuide scheme. As a result, with enough health profile information offered by the medical users, the accuracy of disease risk prediction can be improved. Detailed security analysis shows that our proposed PGuide scheme ensures the privacy-preservation for both medical users and service provider. In addition, the performance evaluation via extensive experiments also demonstrates that our proposed PPCP protocol is much efficient in terms of low computational cost and communication overhead.

show abstract

Prediction models for clustered data: comparison of a random intercept and standard regression model

Cited by 77 publications

References 23 publications

Beyond discrimination: A comparison of calibration methods and clinical usefulness of predictive models of readmission risk

Beyond discrimination: A comparison of calibration methods and clinical usefulness of predictive models of readmission risk

PSLP: Privacy-preserving single-layer perceptron learning for e-Healthcare

PGuide: An Efficient and Privacy-Preserving Smartphone-Based Pre-Clinical Guidance Scheme

Contact Info

Product

Resources

About