A two dimensional accuracy-based measure for classification performance

Carbonero-Ruz, Mariano; Martínez-Estudillo, Francisco José; Fernández-Navarro, Francisco; Becerra-Alonso, David; Martínez-Estudillo, Alfonso Carlos

doi:10.1016/j.ins.2016.12.005

Cited by 10 publications

(8 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To the best of our knowledge, this is the first time that such a wide range of instruments have been reviewed in this scope. As a growing body of literature conducts isolated reviews and uses different types of instruments (either graphical [ 83 , 84 ], probabilistic [ 55 ], or entropic [ 51 ]) and researchers use binary-classification instruments in multi-class classification [ 31 , 38 , 39 ] as well as regression and time-series forecasting applications [ 85 ], such a holistic identification and representation of those instruments is critical to see and analyze the whole as well as to adapt or transfer the knowledge and practices into those applications.…”

Section: Conclusion and Discussionmentioning

confidence: 99%

“…To the best of our knowledge, an extensive analysis of performance evaluation instruments in broad coverage has not been conducted in the literature. This study also provides a baseline for the performance evaluation of classifications with a higher number of classes, because binary-classification evaluation metrics can be used for multi-class or multi-label classification by micro- or macro-averaging binary metrics [ 37 , 38 ] or making specific adaptions such as one- versus -all approach [ 31 , 32 , 39 ]. Moreover, the literature has recently focused on the need for reliable measurement [ 40 ].…”

Section: Case Study: Performance Evaluation In Android Mobile-malware...mentioning

confidence: 99%

See 1 more Smart Citation

PToPI: A Comprehensive Review, Analysis, and Knowledge Representation of Binary Classification Performance Measures/Metrics

2022

View full text Add to dashboard Cite

Although few performance evaluation instruments have been used conventionally in different machine learning-based classification problem domains, there are numerous ones defined in the literature. This study reviews and describes performance instruments via formally defined novel concepts and clarifies the terminology. The study first highlights the issues in performance evaluation via a survey of 78 mobile-malware classification studies and reviews terminology. Based on three research questions, it proposes novel concepts to identify characteristics, similarities, and differences of instruments that are categorized into ‘performance measures’ and ‘performance metrics’ in the classification context for the first time. The concepts reflecting the intrinsic properties of instruments such as canonical form, geometry, duality, complementation, dependency, and leveling, aim to reveal similarities and differences of numerous instruments, such as redundancy and ground-truth versus prediction focuses. As an application of knowledge representation, we introduced a new exploratory table called PToPI (Periodic Table of Performance Instruments) for 29 measures and 28 metrics (69 instruments including variant and parametric ones). Visualizing proposed concepts, PToPI provides a new relational structure for the instruments including graphical, probabilistic, and entropic ones to see their properties and dependencies all in one place. Applications of the exploratory table in six examples from different domains in the literature have shown that PToPI aids overall instrument analysis and selection of the proper performance metrics according to the specific requirements of a classification problem. We expect that the proposed concepts and PToPI will help researchers comprehend and use the instruments and follow a systematic approach to classification performance evaluation and publication.

show abstract

Section: Conclusion and Discussionmentioning

confidence: 99%

Section: Case Study: Performance Evaluation In Android Mobile-malware...mentioning

confidence: 99%

PToPI: A Comprehensive Review, Analysis, and Knowledge Representation of Binary Classification Performance Measures/Metrics

2022

View full text Add to dashboard Cite

show abstract

“…While these classifiers may have varying output formats, they all provide information about the predicted category for each instance. By utilizing this information, we can construct the confusion matrix, which enables us to calculate metrics such as accuracy, precision, recall, and F-score 52 . This approach ensures a consistent and comprehensive evaluation across all classifiers, facilitating meaningful comparisons and providing valuable insights into their performance.…”

Section: Methodsmentioning

confidence: 99%

Artificial intelligence extracts key insights from legal documents to predict intimate partner femicide

Garcia-Vergara,

Almeda,

Fernández-Navarro

et al. 2023

Sci Rep

Self Cite

View full text Add to dashboard Cite

Legal documents serve as valuable repositories of information pertaining to crimes, encompassing not only legal aspects but also relevant details about criminal behaviors. To date and the best of our knowledge, no studies in the field examine legal documents for crime understanding using an Artificial Intelligence (AI) approach. The present study aims to fill this research gap by identifying relevant information available in legal documents for crime prediction using Artificial Intelligence (AI). This innovative approach will be applied to the specific crime of Intimate Partner Femicide (IPF). A total of 491 legal documents related to lethal and non-lethal violence by male-to-female intimate partners were extracted from the Vlex legal database. The information included in these documents was analyzed using AI algorithms belonging to Bayesian, functions-based, instance-based, tree-based, and rule-based classifiers. The findings demonstrate that specific information from legal documents, such as past criminal behaviors, imposed sanctions, characteristics of violence severity and frequency, as well as the environment and situation in which this crime occurs, enable the correct detection of more than three-quarters of both lethal and non-lethal violence within male-to-female intimate partner relationships. The obtained knowledge is crucial for professionals who have access to legal documents, as it can help identify high-risk IPF cases and shape strategies for preventing crime. While this study focuses on IPF, this innovative approach has the potential to be extended to other types of crimes, making it applicable and beneficial in a broader context.

show abstract

“…Micro average of AUC and the maximum average value of the PR curve area are used in two dimensional fashion as described by Carbonero-Ruz et al. [ 43 ].…”

Section: Methodsmentioning

confidence: 99%

Automated supervised learning pipeline for non-targeted GC-MS data analysis

Sirén

Fischer

Vestner

2019

Analytica Chimica Acta: X

View full text Add to dashboard Cite

Non-targeted analysis is nowadays applied in many different domains of analytical chemistry such as metabolomics, environmental and food analysis. Conventional processing strategies for GC-MS data include baseline correction, feature detection, and retention time alignment before multivariate modeling. These techniques can be prone to errors and therefore time-consuming manual corrections are generally necessary. We introduce here a novel fully automated approach to non-targeted GC-MS data processing. This new approach avoids feature extraction and retention time alignment. Supervised machine learning on decomposed tensors of segmented chromatographic raw data signal is used to rank regions in the chromatograms contributing to differentiation between sample classes. The performance of this novel data analysis approach is demonstrated on three published datasets.

show abstract

A two dimensional accuracy-based measure for classification performance

Cited by 10 publications

References 22 publications

PToPI: A Comprehensive Review, Analysis, and Knowledge Representation of Binary Classification Performance Measures/Metrics

PToPI: A Comprehensive Review, Analysis, and Knowledge Representation of Binary Classification Performance Measures/Metrics

Artificial intelligence extracts key insights from legal documents to predict intimate partner femicide

Automated supervised learning pipeline for non-targeted GC-MS data analysis

Contact Info

Product

Resources

About