Association of parental cardiovascular risk factors with offspring type 1 diabetes mellitus insulin sensitivity

Pinto, Camila Albuquerque; Sá, João Roberto de; Lana, Janaína Martins de; Dualib, Patricia; Gabbay, Mônica Andrade Lima; Dib, Sérgio Atala

doi:10.1016/j.jdiacomp.2022.108393

Cited by 3 publications

(1 citation statement)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The use of a perfect, gold standard, reference data set in which all class labels contained are completely correct is often assumed implicitly in an accuracy assessment. However, such a gold standard may not exist [ 37 , 38 ] or might be unavailable because it is perhaps too costly or impractical to use [ 16 , 20 , 39 ]. Such situations force the use of an imperfect reference standard.…”

Section: Introductionmentioning

confidence: 99%

Challenges in the real world use of classification accuracy metrics: From recall and precision to the Matthews correlation coefficient

Foody

2023

PLoS ONE

View full text Add to dashboard Cite

The accuracy of a classification is fundamental to its interpretation, use and ultimately decision making. Unfortunately, the apparent accuracy assessed can differ greatly from the true accuracy. Mis-estimation of classification accuracy metrics and associated mis-interpretations are often due to variations in prevalence and the use of an imperfect reference standard. The fundamental issues underlying the problems associated with variations in prevalence and reference standard quality are revisited here for binary classifications with particular attention focused on the use of the Matthews correlation coefficient (MCC). A key attribute claimed of the MCC is that a high value can only be attained when the classification performed well on both classes in a binary classification. However, it is shown here that the apparent magnitude of a set of popular accuracy metrics used in fields such as computer science medicine and environmental science (Recall, Precision, Specificity, Negative Predictive Value, J, F1, likelihood ratios and MCC) and one key attribute (prevalence) were all influenced greatly by variations in prevalence and use of an imperfect reference standard. Simulations using realistic values for data quality in applications such as remote sensing showed each metric varied over the range of possible prevalence and at differing levels of reference standard quality. The direction and magnitude of accuracy metric mis-estimation were a function of prevalence and the size and nature of the imperfections in the reference standard. It was evident that the apparent MCC could be substantially under- or over-estimated. Additionally, a high apparent MCC arose from an unquestionably poor classification. As with some other metrics of accuracy, the utility of the MCC may be overstated and apparent values need to be interpreted with caution. Apparent accuracy and prevalence values can be mis-leading and calls for the issues to be recognised and addressed should be heeded.

show abstract