Adjusted Geometric-Mean: A Novel Performance Measure for Imbalanced Bioinformatics Datasets Learning

Abstract

One common and challenging problem faced by many bioinformatics applications, such as promoter recognition, splice site prediction, RNA gene prediction, drug discovery and protein classification, is the imbalance of the available datasets. In most of these applications, the positive data examples are largely outnumbered by the negative data examples, which often leads to the development of sub-optimal prediction models having high negative recognition rate (Specificity = SP) and low positive recognition rate (Sensitivity = SE). When class imbalance learning methods are applied, usually, the SE is increased at the expense of reducing some amount of the SP. In this paper, we point out that in these data-imbalanced bioinformatics applications, the goal of applying class imbalance learning methods would be to increase the SE as high as possible by keeping the reduction of SP as low as possible. We explain that the existing performance measures used in class imbalance learning can still produce sub-optimal models with respect to this classification goal. In order to overcome these problems, we introduce a new performance measure called Adjusted Geometric-mean (AGm). The experimental results obtained on ten real-world imbalanced bioinformatics datasets demonstrates that the AGm metric can achieve a lower rate of reduction of SP than the existing performance metrics, when increasing the SE through class imbalance learning methods. This characteristic of AGm metric makes it more suitable for achieving the proposed classification goal in imbalanced bioinformatics datasets learning.

An Event Group Based Classification Framework for Multi-variate Sequential Data

“…Because the BF data is an imbalanced 3-class dataset, in order to objectively evaluate the performance, the Adjusted Geometric Mean (AGM) (Batuwita & Palade 2012) is selected as the main metric for evaluating the prediction performance. The AGM is an extension of the GM (geometric mean) metric, which in turn is calculated from the four basic statistical measures (True Positive (TP), False Positive (FP), True Negative (TN) and False Negative (FN)) with the following formulas:…”

An empirical study of self-training and data balancing techniques for splice site prediction

“…Supervised machine learning approaches for bioinformatics problems have been widely used (Liu et al, 2012;Chen, 2008;Wang and Wu, 2006;Yu et al, 2013;Erdoğdu et al, 2013;Jiang et al, 2013;Rider et al, 2014;Huang, 2013). The problem of identifying splice sites using machine learning techniques has also been addressed, mostly by supervised methods (Baten, et al, 2006;Baten et al, 2007;Sonnenburg et al, 2007;Castelo and Guigó, 2004;Batuwita and Palade, 2012). For example, in , the authors present a state-of-the-art method using SVM and an RBF kernel for human splice site detection.…”

“…Large-margin based classifiers, such as SVM, would be impractical, due to their large number of parameters that need tuning and longer computational times as compared to Naïve Bayes. Similar to Batuwita and Palade (2012), we also used under-and SMOTE over-sampling. Wei and Dunbrack (2013) explored the effects that balancing both the training and test datasets have on the SVM algorithm.…”

“…Similar to other fields, the data imbalance problem has been addressed for many bioinformatics tasks (Wei and Dunbrack, 2013;Batuwita and Palade, 2010;Batuwita and Palade, 2012;Lusa and Blagus, 2010;Yu et al, 2011) in the supervised framework, under the assumption that a sufficiently large amount of labelled data is available. However, in bioinformatics, labelled examples are traditionally obtained via wet-lab experiments, which are expensive and time-consuming methods and necessitate biological know-how.…”

An empirical study of self-training and data balancing techniques for splice site prediction

“…The same authors also proposed a novel measure for evaluating the learning of supervised classifiers on imbalanced bioinformatics datasets, namely the 'adjusted geometric-mean' (Batuwita and Palade, 2012). In this work, the authors conducted experiments on ten DNA (including splice sites) and protein datasets, with imbalance degrees of up to 1-to-14 and dataset sizes with up to 10K instances.…”