An Invitation to Greater Use of Matthews Correlation Coefficient in Robotics and Artificial Intelligence

Chicco, Davide; Jurman, Giuseppe

doi:10.3389/frobt.2022.876814

Cited by 17 publications

(5 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the other hand, MCC stands out as a statistical metric that yields a high score exclusively when all 4 diagnostic efficiency measures exhibit heightened values [ 36 ]. Since MCC offers greater informativeness and reliability, we computed this metric for all 6 risk scores.…”

Section: Resultsmentioning

confidence: 99%

Risk scores for major bleeding from direct oral anticoagulants: comparing predictive performance in patients with atrial fibrillation

Campos-Staffico,

Jacoby,

Dorsch

et al. 2024

Research and Practice in Thrombosis and Haemostasis

View full text Add to dashboard Cite

Section: Resultsmentioning

confidence: 99%

Risk scores for major bleeding from direct oral anticoagulants: comparing predictive performance in patients with atrial fibrillation

Campos-Staffico,

Jacoby,

Dorsch

et al. 2024

Research and Practice in Thrombosis and Haemostasis

View full text Add to dashboard Cite

“…Matthews correlation coefficient (MCC) 21 was chosen as the state-of-the-art overall performance metric as it is robust against the uneven distribution of outcomes in datasets, and it is the most reliable classification metric for considering both positive and negative classes evenly. 22 MCC is more reliable than the Cohen k-statistic, Brier score, balanced accuracy (Bal Acc), and accuracy (Acc), because they are all influenced by the outcome prevalence, which varies across different clinical settings. [21][22][23][24][25][26][27][28][29] Also, it is more clinically relevant than the area under the receiver operating characteristic curve (AUROC) and the area under the precision-recall curve.…”

Section: Methodsmentioning

confidence: 99%

“…22 MCC is more reliable than the Cohen k-statistic, Brier score, balanced accuracy (Bal Acc), and accuracy (Acc), because they are all influenced by the outcome prevalence, which varies across different clinical settings. [21][22][23][24][25][26][27][28][29] Also, it is more clinically relevant than the area under the receiver operating characteristic curve (AUROC) and the area under the precision-recall curve. 23,30 If not explicitly reported, MCC was calculated as per 21,25 (Equation S3.1).…”

Section: Methodsmentioning

confidence: 99%

Opportunities and challenges for the application of artificial intelligence paradigms into the management of endemic viral infections: The example of Chronic Hepatitis C Virus

Farrag,

Kamel,

El‐Baraky

2024

Reviews in Medical Virology

View full text Add to dashboard Cite

Despite the advent of direct‐acting antiviral agents (DAAs) as a definitive therapy for chronic hepatitis C virus (HCV) infection, the burden of the disease remains globally elevated. The emerging big data on different HCV paradigms fostered the introduction of artificial intelligence/machine learning (AI/ML) applications to help decrease that burden by providing more optimised strategies for early diagnosis and treatment prioritisation. The current review provides descriptive and analytical insight into the recently published AI/ML applications in five medical aspects of HCV infection. In addition, it highlights the opportunities these powerful tools offer in designing national health policies that prioritise HCV patients for the costly DAAs and developing broadly neutralising HCV antibodies. Finally, this paper highlights the challenges encountered in developing and applying these AI/ML models to clinical practice and suggests schemes to overcome some of them. The presented models were primarily evaluated using the Matthews correlation coefficient and the F1‐score to make a more reliable inference about their predictive power under imbalanced datasets. Many published AI/ML applications offered great utilities for predicting novel HCV treatments and prioritising patients for DAAs receipt, especially in settings of limited resources and high HCV burden. Some outperformed the classical diagnostic tools, such as third‐generation serological tests, alpha‐fetoprotein, and ultrasound, in detecting HCV infections and early HCV‐associated hepatocellular carcinoma, respectively. However, further statistical and clinical validation of AI/ML models is highly advocated before incorporating these applications into clinical practice.

show abstract

“…Hence, we chose MCC over Cohen's kappa. An MCC that is well above zero indicates a performance that is better than random guessing [63]. A more detailed explanation of the three performance metrics and a brief discussion on the choice between MCC and Cohen's kappa are attached in the Appendix.…”

Section: Methodsmentioning

confidence: 99%

Affect Recognition in Muscular Response Signals

et al. 2023

View full text Add to dashboard Cite

This study investigated the potential of recognising arousal in motor activity collected by wristworn accelerometers. We hypothesise that emotional arousal emerges from the generalised central nervous system which embeds affective states within motor activity. We formulate arousal detection as a statistical problem of separating two sets -motor activity under emotional arousal and motor activity without arousal. We propose a novel test regime based on machine learning assuming that the two sets can be distinguished if a machine learning classifier can separate the sets better than random guessing. To increase the statistical power of the testing regime, the performance of the classifiers is evaluated in a cross-validation framework, and to test if the classifiers perform better than random guessing, a repeated cross-validation corrected t-test is used. The classifiers were evaluated on the basis of accuracy and Matthew's correlation coefficient. The suggested procedures were further compared against a traditional multivariate paired Hotelling's T-squared test. The classifiers achieved an accuracy of about 60%, and according to the proposed t-test were significantly better than random guessing. The suggested test regime demonstrated higher statistical power than Hotelling's T-squared test, and we conclude that we can distinguish between motor activity under emotional arousal and without it.

show abstract

An Invitation to Greater Use of Matthews Correlation Coefficient in Robotics and Artificial Intelligence

Cited by 17 publications

References 25 publications

Risk scores for major bleeding from direct oral anticoagulants: comparing predictive performance in patients with atrial fibrillation

Risk scores for major bleeding from direct oral anticoagulants: comparing predictive performance in patients with atrial fibrillation

Opportunities and challenges for the application of artificial intelligence paradigms into the management of endemic viral infections: The example of Chronic Hepatitis C Virus

Affect Recognition in Muscular Response Signals

Contact Info

Product

Resources

About