ROC curves and nonrandom data

Cook, Jonathan

doi:10.1016/j.patrec.2016.11.015

Cited by 28 publications

(16 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Table 2 shows how ROC AUC varies with the portion of positive cases for a given classifier strength. The AUC is calculated analytically using a procedure similar to that of Cook (2017). 4 Table 2 also provides the percent increase in AUC over 50% compared with the AUC when half of the cases are positive.…”

Section: Review Of Pr and Roc Curvesmentioning

confidence: 99%

“…This can lead to big differences in ROC AUC for small changes where the positive cases lie in the ranked list. Also, small changes in the false-positive rate indicate large changes in the number of FP when there are many negatives (this has been discussed by Davis and Goadrich [2006] and Saito and Rehmsmeier [2015]). The second issue is that, in many settings, ROC AUC is increasing in class skew.…”

Section: When Pr Curves Should Be Consultedmentioning

confidence: 99%

“…ROC curves are more attuned to the former and PR to the latter. Davis and Goadrich (2006) provide an example that highlights the different information provided by ROC and PR curves. We reproduce this example in figure 1.…”

Section: When Pr Curves Should Be Consultedmentioning

confidence: 99%

“…Figure 1. An example from Davis and Goadrich (2006) that illustrates how ROC and PR curves can provide different information about a classifier's performance…”

Section: When Pr Curves Should Be Consultedmentioning

confidence: 99%

See 3 more Smart Citations

When to consult precision-recall curves

Cook¹,

Ramadas²

2020

The Stata Journal: Promoting communications on statistics and S

View full text Add to dashboard Cite

Receiver operating characteristic (ROC) curves are commonly used to evaluate predictions of binary outcomes. When there is a small percentage of items of interest (as would be the case with fraud detection, for example), ROC curves can provide an inflated view of performance. This can cause challenges in determining which set of predictions is better. In this article, we discuss the conditions under which precision-recall curves may be preferable to ROC curves. As an illustrative example, we compare two commonly used fraud predictors (Beneish’s [1999, Financial Analysts Journal 55: 24–36] M score and Dechow et al.’s [2011, Contemporary Accounting Research 28: 17–82] F score) using both ROC and precision-recall curves. To aid the reader with using precision-recall curves, we also introduce the command prcurve to plot them.

show abstract

Section: Review Of Pr and Roc Curvesmentioning

confidence: 99%

Section: When Pr Curves Should Be Consultedmentioning

confidence: 99%

Section: When Pr Curves Should Be Consultedmentioning

confidence: 99%

“…Figure 1. An example from Davis and Goadrich (2006) that illustrates how ROC and PR curves can provide different information about a classifier's performance…”

Section: When Pr Curves Should Be Consultedmentioning

confidence: 99%

See 2 more Smart Citations

When to consult precision-recall curves

Cook¹,

Ramadas²

2020

The Stata Journal: Promoting communications on statistics and S

View full text Add to dashboard Cite

show abstract

“…The ROC curve (Cook, 2017) is an evaluation index commonly used in diagnostic tests. In general, Area under Curve (AUC) is often used as a criterion.…”

Section: Resultsmentioning

confidence: 99%

An Imitation medical diagnosis method of hydro-turbine generating unit based on Bayesian network

Cheng

Zhu

et al. 2019

Transactions of the Institute of Measurement and Control

View full text Add to dashboard Cite

In order to improve the intelligent level of fault diagnosis and condition maintenance of hydropower units, an Imitation medical diagnosis method (IMDM) is proposed in this study. IMDM uses Bayesian networks (BN) as the technical framework, including three components: machine learning BN model, expert empirical BN model, and maintenance decision model. Its characteristics are as follows: (i) the machine learning model uses a new node selection method to solve the problem that the traditional fault diagnosis model is difficult to connect with the state monitoring system. (ii) The expert experience BN model improves the traditional method: using the fault tree model to transform the BN structure, Noisy-Or model to simplify conditional probability table, and fuzzy comprehensive evaluation method to obtain the conditional probability. (iii) By introducing the expected utility theory, a maintenance decision model is innovated, which makes sure the optimal maintenance decision scheme after the fault can be better selected. The performance of this proposed method is evaluated by using the experimental data. The results show that the accuracy of the fault reasoning model is higher than 80%, and the maintenance decision model successfully selects 236 optimal maintenance decision schemes from 3159 schemes generated by 13 faults.

show abstract