Comparing predictions made by a prediction model, clinical score, and physicians

Farion, Ken; Wilk, Szymon; Michalowski, Wojtek; O’Sullivan, Dympna; Sayyad-Shirabad, Jelber

doi:10.4338/aci-2013-04-ra-0029

Cited by 41 publications

(33 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, the best performance on both original and preprocessed data sets was observed for the naive Bayes classifier. These findings are consistent with our experience with using naive Bayes alone [7], combining random undersampling with naive Bayes [11,12], as well as with other experimental and theoretical studies on random undersampling [5,6,23] and naive Bayes [18].…”

Section: Discussionsupporting

confidence: 81%

Application of Preprocessing Methods to Imbalanced Clinical Data: An Experimental Study

Wilk

Stefanowski

Wojciechowski

et al. 2016

Advances in Intelligent Systems and Computing

View full text Add to dashboard Cite

In this paper we describe an experimental study where we analyzed data difficulty factors encountered in imbalanced clinical data sets and examined how selected data preprocessing methods were able to address these factors. We considered five data sets describing various pediatric acute conditions. In all these data sets the minority class was sparse and overlapped with the majority classes, thus difficult to learn. We studied five different preprocessing methods: random under-and oversampling, SMOTE, neighborhood cleaning rule and SPIDER2 that were combined with the following classifiers: k-nearest neighbors, decision trees and rules, naive Bayes, neural networks and support vector machines. Application of preprocessing always improved classification performance, and the largest improvement was observed for random undersampling. Moreover, naive Bayes was the best performing classifier regardless of a used preprocessing method.

show abstract

Section: Discussionsupporting

confidence: 81%

Application of Preprocessing Methods to Imbalanced Clinical Data: An Experimental Study

Wilk

Stefanowski

Wojciechowski

et al. 2016

Advances in Intelligent Systems and Computing

View full text Add to dashboard Cite

show abstract

“…Systems can be passive (users explicitly make a request for support), semi-active (watchdog systems are invoked automatically and present information when users request it) or active (triggered automatically, present information without it being requested and, in some cases, make decisions without the intervention of clinicians). They have been implemented to support clinicians across the spectrum of medical specialties and have been customised for different levels of clinical expertise from novice (eg student nurses) 10 through nonspecialist (eg in community hospital settings) 11 to highly specialist healthcare professionals (eg digital pathology). 12 Simple CDSSs usually check the input provided by a clinician and verify whether the value is allowable or within a specifi ed range and whether there are any predefi ned contraindications.…”

Section: Brief Taxonomy Of Clinical Decision Support Systemsmentioning

confidence: 99%

Decision time for clinical decision support systems

et al. 2014

View full text Add to dashboard Cite

KEYWORDS: Clinical decision support systems, decision-making, diagnosis ContextInformation technology (IT) is now commonplace in almost every branch of healthcare. Electronic health records, eprescribing and digital medical imaging are well known to clinicians and have been implemented with varying degrees of success. 1 In addition, clinicians increasingly make use of online repositories such as PubMed and Google Scholar, 2 and specialised search engines such as FindZebra 3 to help answer clinical questions. One often overlooked set of IT tools are clinical decision support systems (CDSSs), which have been defi ned as systems that 'provide clinicians or patients with computer-generated clinical knowledge and patient-related information, intelligently fi ltered or presented at appropriate times, to enhance patient care'. 4 Such CDSSs have been the subject of academic computer science research for more than ABSTRACT 50 years 5 and offer the potential for better supported decision making by clinicians, improved compliance with medical standards and improved clinical effi ciency and safety. 6,7 Nonetheless, utilisation of CDSSs remains limited, and most healthcare IT systems do not include robust CDSS functions that can be widely employed across organisations, clinical presentations and domains. 8 Some of the challenges to implementation of CDSSs relate to the volume of high-quality data required for state-of-the-art systems, the translation of such data to machine-readable states and the mapping of CDSS processes to fi t with existing clinical workfl ows. As a result, successful implementation of CDSSs has tended to be site and domain specifi c, with major diffi culties replicating these successes more extensively throughout healthcare systems. 9 This is in contrast to commercial fi elds such as fi nance, where decision support technologies have been widely deployed. For example, risk-profi ling tools for fi nancial experts have been developed as easy-to-use programs that can assimilate information and guide users through complex fi nancial information and associated decisions tailored to individual customer needs. Healthcare decision making is signifi cantly more complex than fi nancial planning; however, some of the challenges in both domains are similar: large quantities of data need to be linked, integrated and translated to machine readable formats, and expert knowledge is required to contextualise and apply the data in a meaningful way. We discuss some reasons for the limited dissemination and adoption of CDSSs to date and refl ect on the major barriers that need to be overcome for these useful tools to be adopted more widely.

show abstract

“…[22] Further validation of the PRAM in 782 children ages 2–17 was published as the Pediatric Respiratory Assessment Measure[23] with measurement of inter-rater reliability, predictive ability for hospitalization before initial treatment (c statistic 0.78), after initial bronchodilation (c statistic 0.84), and using a combination of the initial PRAM and change in PRAM after initial treatment (c statistic 0.86). The PRAM has been shown in other populations to be predictive of hospitalization[24] and has been compared directly to PASS,[16] a Bayesian predictive model, clinician prediction,[25] and spirometry. [26]…”

Section: Methodsmentioning

confidence: 99%

Direct concurrent comparison of multiple pediatric acute asthma scoring instruments

et al. 2017

View full text Add to dashboard Cite

Objective Appropriate delivery of Emergency Department (ED) treatment to children with acute asthma requires clinician assessment of acute asthma severity. Various clinical scoring instruments exist to standardize assessment of acute asthma severity in the ED, but their selection remains arbitrary due to few published direct comparisons of their properties. Our objective was to test the feasibility of directly comparing properties of multiple scoring instruments in a pediatric ED. Methods Using a novel approach supported by a composite data collection form, clinicians categorized elements of 5 scoring instruments before and after initial treatment for 48 patients 2–18 years of age with acute asthma seen at the emergency department of a tertiary care pediatric hospital ED from August to December 2014. Scoring instruments were compared for inter-rater reliability between clinician types and their ability to predict hospitalization. Results Inter-rater reliability between clinician types was not different between instruments at any point and was lower (weighted kappa range 0.21 – 0.55) than values reported elsewhere. Predictive ability of most instruments for hospitalization was higher after treatment than before treatment (p<0.05) and may vary between instruments after treatment (p=0.054). Conclusions We demonstrate the feasibility of comparing multiple clinical scoring instruments simultaneously in ED clinical practice. Scoring instruments had higher predictive ability for hospitalization after treatment than before treatment and may differ in their predictive ability after initial treatment. Definitive conclusions about the best instrument or meaningful comparison between instruments will require a study with a larger sample size.

show abstract

Comparing predictions made by a prediction model, clinical score, and physicians

Cited by 41 publications

References 26 publications

Application of Preprocessing Methods to Imbalanced Clinical Data: An Experimental Study

Application of Preprocessing Methods to Imbalanced Clinical Data: An Experimental Study

Decision time for clinical decision support systems

Direct concurrent comparison of multiple pediatric acute asthma scoring instruments

Contact Info

Product

Resources

About