Receiver Operating Characteristic (ROC) Curve

서동석,; 이상용,; 박세광,; 노주헌,

doi:10.4135/9781412953948.n394

Cited by 3 publications

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Receiver operating characteristic (ROC) curves were constructed separately for parent and teacher CFM-7 responses to determine the Area Under the ROC Curve (AUC). ROC curves are constructed by plotting the false-positive rate (1—specificity) against the true-positive rate (sensitivity) at each cut-off value defined by the CFM and then drawing a line from x = 0, y = 0 through the values at each cut-off point; the AUC is an overall figure of diagnostic accuracy with a perfect test having a value of 1.0 and a value of 0.5 suggesting that the test result is no better than chance [33,34]. AUC interpretations were classified as excellent (0.96–1.0), very good (0.9 to <0.96), good (0.8 to <0.9), fair (0.7 to <0.8), poor (0.6 to <0.7), and useless (0.5 to <0.6) [33].…”

Section: Methodsmentioning

confidence: 99%

The UNICEF/Washington Group Child Functioning Module—Accuracy, Inter-Rater Reliability and Cut-Off Level for Disability Disaggregation of Fiji’s Education Management Information System

Sprunt

McPake

Marella

2019

IJERPH

View full text Add to dashboard Cite

This paper explores the validity (sensitivity and specificity) of different cut-off levels of the UNICEF/Washington Group Child Functioning Module (CFM) and the inter-rater reliability between teachers and parents as proxy respondents, for disaggregating Fiji’s education management information system (EMIS) by disability. The method used was a cross-sectional diagnostic accuracy study comparing CFM items to standard clinical assessments for 472 primary school aged students in Fiji. Whilst previous domain-specific results showed “good” to “excellent” accuracy of the CFM domains seeing, hearing, walking and speaking, newer analysis shows only “fair” to “poor” accuracy of the cognitive domains (learning, remembering and focusing attention) and “fair” of the overall CFM (area under the Receiver Operating Characteristic curve: 0.763 parent responses, 0.786 teacher responses). Severe impairments are reported relatively evenly across CFM response categories “some difficulty”, “a lot of difficulty” and “cannot do at all”. Most moderate impairments are reported as “some difficulty”. The CFM provides a core component of data required for disaggregating Fiji’s EMIS by disability. However, choice of cut-off level and mixture of impairment severity reported across response categories are challenges. The CFM alone is not accurate enough to determine funding eligibility. For identifying children with disabilities, the CFM should be part of a broader data collection including learning and support needs data and undertaking eligibility verification visits.

show abstract

Section: Methodsmentioning

confidence: 99%

The UNICEF/Washington Group Child Functioning Module—Accuracy, Inter-Rater Reliability and Cut-Off Level for Disability Disaggregation of Fiji’s Education Management Information System

Sprunt

McPake

Marella

2019

IJERPH

View full text Add to dashboard Cite

show abstract

“…The area under the ROC curve, in the current work, was used to compare the efficiency of models. This method has been widely used in recent years to evaluate machine learning algorithms [27]; this method has also been used in the field of medicine, as an effective method, to evaluate the performance of diagnostic tests against the gold standard [28]. Following the modeling procedure based on ANN and GEP, the model obtained from each technique was compared against the AUC value in an attempt to select the best models and techniques.…”

Section: Introductionmentioning

confidence: 99%

Comparing the Efficiency of Artificial Neural Network and Gene Expression Programming in Predicting Coronary Artery Disease

Moghaddasi¹,

mahmoudi²,

Sajadi³

2017

J Health Med Inform

View full text Add to dashboard Cite

Background: Angiography, as the gold standard for the diagnosis of coronary artery disease, has made an attempt to predict coronary artery disease by comparing the efficiency of gene expression programming, as a new data mining technique, and artificial neural network, as a conventional technique. Besides, the study went further to present the results of feature selection based on stepwise backward elimination, classification and regression tree. Methods:The subjects were assessed for nine coronary artery disease risk factors to develop a prediction model for the disease. They included 13,288 patients who were chosen to undergo angiography for the diagnosis of coronary artery disease; from this sample, 4059 subjects were free from the disease while 9169 were suffering from it. Modeling was carried out based on gene expression programming and artificial neural network techniques. The Delong's test was then used to choose the final model based on the area under the Receiver Operating Characteristic (ROC) curve. Results:The model, developed based on artificial neural network, had AUC of 0.719, accuracy of 73.39%, sensitivity of 93.44% and specificity of 28.34%. On the other hand, the model, formulated based on gene expression programming, had AUC of 0.720, accuracy of 73.94%, sensitivity of 93.29% and specificity of 31.43%. Delong's test showed no significant difference between the two models (p value=0/789). Then, feature selection method was used to choose a model with four risk factors and an accuracy rate of 73.26%. Conclusion:Comparison of the results showed no significant difference between the two modeling techniques. The gene expression programming model was very easy to present and interpret; it could also be easily converted to other programming languages; so, with these features in mind, the researchers preferred to choose this technique. modeling, some setting initials are necessary, as can be seen in Table 2. Modeling based on ANN was done using a Multilayer Perceptron (MLP) neural network. Also, the Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithm, developed based on a quasi-Newton algorithm, was used for learning the network. This learning algorithm has a faster convergence rate than the gradient descend and the conjugate gradient algorithms and is one of the appropriate learning algorithms [26]. Since there is no equation for estimating parameters such as the number of neurons in the hidden layer, the layer activation function and error function of a neural network model could be adopted. So, with this point in mind, we created 100 neural network models by randomly selecting the parameter value, as can be seen in [29,30]. In line with this procedure, the stepwise backward elimination method was adopted to compare the results of ANN and GEP and to select the best possible model and technique. As such, the least important risk factors were also removed and the modeling process was carried out with the remaining risk factors. This process continued until there was no significant change in th...

show abstract

Identifying Myocardial Infarction Using Hierarchical Template Matching–Based Myocardial Strain: Algorithm Development and Usability Study (Preprint)

Bhalodiya¹,

Palit²,

Giblin³

et al. 2020

Preprint

View full text Add to dashboard Cite

BACKGROUND Myocardial infarction (MI; location and extent of infarction) can be determined by late enhancement cardiac magnetic resonance (CMR) imaging, which requires the injection of a potentially harmful gadolinium-based contrast agent (GBCA). Alternatively, emerging research in the area of myocardial strain has shown potential to identify MI using strain values. OBJECTIVE This study aims to identify the location of MI by developing an applied algorithmic method of circumferential strain (CS) values, which are derived through a novel hierarchical template matching (HTM) method. METHODS HTM-based CS H-spread from end-diastole to end-systole was used to develop an applied method. Grid-tagging magnetic resonance imaging was used to calculate strain values in the left ventricular (LV) myocardium, followed by the 16-segment American Heart Association model. The data set was used with k-fold cross-validation to estimate the percentage reduction of H-spread among infarcted and noninfarcted LV segments. A total of 43 participants (38 MI and 5 healthy) who underwent CMR imaging were retrospectively selected. Infarcted segments detected by using this method were validated by comparison with late enhancement CMR, and the diagnostic performance of the applied algorithmic method was evaluated with a receiver operating characteristic curve test. RESULTS The H-spread of the CS was reduced in infarcted segments compared with noninfarcted segments of the LV. The reductions were 30% in basal segments, 30% in midventricular segments, and 20% in apical LV segments. The diagnostic accuracy of detection, using the reported method, was represented by area under the curve values, which were 0.85, 0.82, and 0.87 for basal, midventricular, and apical slices, respectively, demonstrating good agreement with the late-gadolinium enhancement–based detections. CONCLUSIONS The proposed applied algorithmic method has the potential to accurately identify the location of infarcted LV segments without the administration of late-gadolinium enhancement. Such an approach adds the potential to safely identify MI, potentially reduce patient scanning time, and extend the utility of CMR in patients who are contraindicated for the use of GBCA.

show abstract

Receiver Operating Characteristic (ROC) Curve

Cited by 3 publications

References 0 publications

The UNICEF/Washington Group Child Functioning Module—Accuracy, Inter-Rater Reliability and Cut-Off Level for Disability Disaggregation of Fiji’s Education Management Information System

The UNICEF/Washington Group Child Functioning Module—Accuracy, Inter-Rater Reliability and Cut-Off Level for Disability Disaggregation of Fiji’s Education Management Information System

Comparing the Efficiency of Artificial Neural Network and Gene Expression Programming in Predicting Coronary Artery Disease

Identifying Myocardial Infarction Using Hierarchical Template Matching–Based Myocardial Strain: Algorithm Development and Usability Study (Preprint)

Contact Info

Product

Resources

About