Background: There is much interest in the use of prognostic and diagnostic prediction models in all areas of clinical medicine. The use of machine learning to improve prognostic and diagnostic accuracy in this area has been increasing at the expense of classic statistical models. Previous studies have compared performance between these two approaches but their findings are inconsistent and many have limitations. We aimed to compare the discrimination and calibration of seven models built using logistic regression and optimised machine learning algorithms in a clinical setting, where the number of potential predictors is often limited, and externally validate the models. Methods: We trained models using logistic regression and six commonly used machine learning algorithms to predict if a patient diagnosed with diabetes has type 1 diabetes (versus type 2 diabetes). We used seven predictor variables (age, BMI, GADA islet-autoantibodies, sex, total cholesterol, HDL cholesterol and triglyceride) using a UK cohort of adult participants (aged 18-50 years) with clinically diagnosed diabetes recruited from primary and secondary care (n = 960, 14% with type 1 diabetes). Discrimination performance (ROC AUC), calibration and decision curve analysis of each approach was compared in a separate external validation dataset (n = 504, 21% with type 1 diabetes). Results: Average performance obtained in internal validation was similar in all models (ROC AUC ≥ 0.94). In external validation, there were very modest reductions in discrimination with AUC ROC remaining ≥ 0.93 for all methods. Logistic regression had the numerically highest value in external validation (ROC AUC 0.95). Logistic regression had good performance in terms of calibration and decision curve analysis. Neural network and gradient boosting machine had the best calibration performance. Both logistic regression and support vector machine had good decision curve analysis for clinical useful threshold probabilities.

No abstract

Neural mass models (NMMs) are increasingly used to uncover the large-scale mechanisms of brain rhythms in health and disease. The dynamics of these models is dependent upon the choice of parameters, and therefore it is crucial to be able to understand how dynamics change when parameters are varied. Despite being considered low dimensional in comparison to micro-scale, neuronal network models, with regards to understanding the relationship between parameters and dynamics, NMMs are still prohibitively high dimensional for classical approaches such as numerical continuation. Therefore, we need alternative methods to characterise dynamics of NMMs in high dimensional parameter spaces. Here, we introduce a statistical framework that enables the efficient exploration of the relationship between model parameters and selected features of the simulated, emergent model dynamics of NMMs. We combine the classical machine learning approaches of trees and random forests to enable studying the effect that varying multiple parameters has on the dynamics of a model. The method proceeds by using simulations to transform the mathematical model into a database. This database is then used to partition parameter space with respect to dynamic features of interest, using random forests. This allows us to rapidly explore dynamics in high dimensional parameter space, capture the approximate location of qualitative transitions in dynamics and assess the relative importance of all parameters in the model in all dimensions simultaneously. We apply this method to a commonly used NMM in the context of transitions to seizure dynamics. We find that the inhibitory sub-system is most crucial for the generation of seizure dynamics, confirm and expand previous findings regarding the ratio of excitation and inhibition, and demonstrate that previously overlooked parameters can have a significant impact on model dynamics. We advocate the use of this method in future to constrain high dimensional parameter spaces enabling more efficient, person-specific, model calibration.

OBJECTIVE Genetic risk scores (GRS) aid classification of diabetes type in White European adult populations. We aimed to assess the utility of GRS in the classification of diabetes type among racially/ethnically diverse youth in the U.S. RESEARCH DESIGN AND METHODS We generated type 1 diabetes (T1D)- and type 2 diabetes (T2D)-specific GRS in 2,045 individuals from the SEARCH for Diabetes in Youth study. We assessed the distribution of genetic risk stratified by diabetes autoantibody positive or negative (DAA+/−) and insulin sensitivity (IS) or insulin resistance (IR) and self-reported race/ethnicity (White, Black, Hispanic, and other). RESULTS T1D and T2D GRS were strong independent predictors of etiologic type. The T1D GRS was highest in the DAA+/IS group and lowest in the DAA−/IR group, with the inverse relationship observed with the T2D GRS. Discrimination was similar across all racial/ethnic groups but showed differences in score distribution. Clustering by combined genetic risk showed DAA+/IR and DAA−/IS individuals had a greater probability of T1D than T2D. In DAA− individuals, genetic probability of T1D identified individuals most likely to progress to absolute insulin deficiency. CONCLUSIONS Diabetes type–specific GRS are consistent predictors of diabetes type across racial/ethnic groups in a U.S. youth cohort, but future work needs to account for differences in GRS distribution by ancestry. T1D and T2D GRS may have particular utility for classification of DAA− children.