Bootstrapping the out-of-sample predictions for efficient and accurate cross-validation

Tsamardinos, Ioannis; Greasidou, Elissavet; Borboudakis, Giorgos

doi:10.1007/s10994-018-5714-4

Cited by 165 publications

(159 citation statements)

References 42 publications

Supporting

Mentioning

155

Contrasting

Unclassified

Order By: Relevance

“…44 for an explanation); the optimism is removed using a bootstrap procedure before it is returned in a similar fashion as in ref. 44 In the end, K performance estimates are computed for each configuration, and the one with the best average performance is selected as the best configuration. A final model is produced by applying the best configuration on the complete set of MOFs.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Chemically intuited, large-scale screening of MOFs by machine learning techniques

et al. 2017

Self Cite

View full text Add to dashboard Cite

A novel computational methodology for large-scale screening of MOFs is applied to gas storage with the use of machine learning technologies. This approach is a promising trade-off between the accuracy of ab initio methods and the speed of classical approaches, strategically combined with chemical intuition. The results demonstrate that the chemical properties of MOFs are indeed predictable (stochastically, not deterministically) using machine learning methods and automated analysis protocols, with the accuracy of predictions increasing with sample size. Our initial results indicate that this methodology is promising to apply not only to gas storage in MOFs but in many other material science projects.npj Computational Materials (2017) 3:40 ; doi:10.1038/s41524-017-0045-8 INTRODUCTION Metal-organic frameworks (MOFs) or porous coordination polymers are a rapidly growing family of hybrid inorganic-organic nanoporous materials, which belong to the category of coordination polymers.1-3 These relatively new materials consist of a threedimensional periodic network, constructed from molecular building blocks, such as metal clusters and organic linkers (Fig. 1). The possible combinations of these numerous building blocks under different topologies result is an almost unlimited number of potential MOFs! Since their discovery 4 MOFs have attracted significant scientific attention due to their extraordinary properties. As "skeleton" materials, they pose very large pores and outstanding apparent surface area. If we were able to unwrap the surface of only one gram of these "very empty" materials, we could cover the area of a football court! These unique characteristics of the MOFs made them excellent candidates for catalysis and gas storage applications.MOFs have shown exceptional performance in gas storage and separation. Both useful and harmful gases can be absorbed in their pores in very large amounts. The storage of hydrogen,

show abstract

Section: Methodsmentioning

confidence: 99%

“…The optimism problem has been noted both theoretically as well as experimentally. JAD estimates the bias of the performance using a bootstrap method, 43,44 and removes it to return the final performance estimate.…”

Section: Methodsmentioning

confidence: 99%

Chemically intuited, large-scale screening of MOFs by machine learning techniques

et al. 2017

Self Cite

View full text Add to dashboard Cite

show abstract

“…On this dataset, we performed nested-cross validation by combining a three-way split of the data (trainingvalidation-testing) with leave-one-out cross-validation (CV) and grid search for SVM parameter (boxconstraint) tuning. This was done to avoid upward bias in the metrics of performance estimates (Guyon and Elisseeff, 2003;Tsamardinos et al, 2018). Additionally, we avoided any bias in the selection of the most discriminatory threshold pair (i.e., z-score and percentage abnormality) to determine the node abnormality by computing it at every step of cross-validation after removing the test subject (Smialowski et al, 2009).…”

Section: Predictive Model Design For Generalizability Assessmentmentioning

confidence: 99%

Node abnormality predicts seizure outcome and relates to long-term relapse after epilepsy surgery

Sinha

Wang

Silva

et al. 2019

Preprint

View full text Add to dashboard Cite

Objective:We assessed pre-operative structural brain networks and clinical characteristics of patients with drug resistant temporal lobe epilepsy (TLE) to identify correlates of post-surgical seizure outcome at 1 year and seizure relapses up to 5 years. Methods:We retrospectively examined data from 51 TLE patients who underwent anterior temporal lobe resection (ATLR) and 29 healthy controls. For each patient, using the pre-operative structural, diffusion, and post-operative structural MRI, we generated two networks: 'presurgery' network and 'surgically-spared' network. The pre-surgery network is the whole-brain network before surgery and the surgically-spared network is a subnetwork of the pre-surgery network which is expected to remain unaffected by surgery and hence present post-operatively.Standardising these networks with respect to controls, we determined the number of abnormal nodes before surgery and expected to remain after surgery. We incorporated these 2 abnormality measures and 13 commonly acquired clinical data from each patient in a robust machine learning framework to estimate patient-specific chances of seizures persisting after surgery. Results:Patients with more abnormal nodes had lower chance of seizure freedom at 1 year and even if seizure free at 1 year, were more likely to relapse within five years. In the surgically-spared networks of poor outcome patients, the number of abnormal nodes was greater and their locations more widespread than in good outcome patients. We achieved 0.84 ± 0.06 AUC and 0.89 ± 0.09 specificity in detecting unsuccessful seizure outcomes at 1-year. Moreover, the model-predicted likelihood of seizure relapse was significantly correlated with the grade of surgical outcome at year-one and associated with relapses up-to five years post-surgery. Conclusion:Node abnormality offers a personalised non-invasive marker, that can be combined with clinical data, to better estimate the chances of seizure freedom at 1 year, and subsequent relapse up to 5 years after ATLR.

show abstract

“…To do so, the sample was randomly divided into three mutually exclusive datasets: Training (70%), validation (10%), and testing (20%). This process used the 10 fold cross-validation method with 500 iterations to estimate error ratios [37]. The first subset of data was used to train the models and estimating the parameters.…”

Section: Research Stepsmentioning

confidence: 99%

Currency Crises Prediction Using Deep Neural Decision Trees

et al. 2019

View full text Add to dashboard Cite

Featured Application: The superiority of a novel computational technique (deep neural decision trees) for prediction of currency crises over other methodologies and the construction of new crisis prediction models more precise than existing ones.Abstract: Currency crises are major events in the international monetary system. They affect the monetary policy of countries and are associated with risks of vulnerability for open economies. Much research has been carried out on the behavior of these events, and models have been developed to predict falls in the value of currencies. However, the limitations of existing models mean further research is required in this area, since the models are still of limited accuracy and have only been developed for emerging countries. This article presents an innovative global model for predicting currency crises. The analysis is geographically differentiated for regions, considering both emerging and developed countries and can accurately estimate future scenarios for currency crises at the global level. It uses a sample of 162 countries making it possible to account for the regional heterogeneity of the warning indicators. The method used was deep neural decision trees (DNDTs), a technique based on decision trees implemented by deep learning neural networks, which was compared with other methodologies widely applied in prediction. Our model has significant potential for the adaptation of macroeconomic policy to the risks derived from falls in the value of currencies, providing tools that help ensure financial stability at the global level. Appl. Sci. 2019, 9, 5227 2 of 18 effect of contagion on other countries. However, the event was about more than just speculation on the Thai currency and saw the collapse of Asian stock markets. The financial crisis that began with the devaluation of the Thai baht exchange rate resulted in a sharp increase in interest rates and the collapse of many companies, as well as an increase in the cost of credit and a general fall in GDP in the region [1]. This resulted in foreign and national investors pulling out investment. Not only did this crisis affect Asian countries, but it also had a negative impact on other emerging economies, especially in Latin America, showing that currency crises are not limited to a specific economy. Globalization can increase the economic difficulties of societies and affect the structure of national economies after the real economy has suffered a damaging impact [2]. Figure 1 shows the number of currency crises per year at the international level. Appl. Sci. 2019, 8, x FOR PEER REVIEW 2 of 19of contagion on other countries. However, the event was about more than just speculation on the Thai currency and saw the collapse of Asian stock markets. The financial crisis that began with the devaluation of the Thai baht exchange rate resulted in a sharp increase in interest rates and the collapse of many companies, as well as an increase in the cost of credit and a general fall in GDP in the region [1]. This resulted in foreig...

show abstract

Bootstrapping the out-of-sample predictions for efficient and accurate cross-validation

Cited by 165 publications

References 42 publications

Chemically intuited, large-scale screening of MOFs by machine learning techniques

Chemically intuited, large-scale screening of MOFs by machine learning techniques

Node abnormality predicts seizure outcome and relates to long-term relapse after epilepsy surgery

Currency Crises Prediction Using Deep Neural Decision Trees

Contact Info

Product

Resources

About