An Evaluation of Classification Algorithms Using Mc Nemar’s Test

Bostanci, Betul; Bostancı, Erkan

doi:10.1007/978-81-322-1038-2_2

Cited by 53 publications

(56 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Further, for both feature sets, features extracted 320 from the non-speech parts yield higher identification rates than features extracted from the whole utterance or the speech parts. For each classifier, the performance differences between the baseline features (extracted from the speech parts) and those extracted from the whole utterance or the non-speech parts were found to be significantly different according to McNemar's test [48]. McNemar's test tabulates the correlation of the correct 325 and incorrect decisions between two systems and counts the number of trials that two systems disagree and then uses the chi-square test statistics to compute the p-values.…”

Section: Comparison Of Mfcc and Lfcc Featuresmentioning

confidence: 99%

Source cell-phone recognition from recorded speech using non-speech segments

Hanilçi

Kinnunen

2014

Digital Signal Processing

View full text Add to dashboard Cite

In a recent study, we have introduced the problem of identifying cell-phones using recorded speech and shown that speech signals convey information about the source device, making it possible to identify the source with some accuracy. In this paper, we consider recognizing source cell-phone microphones using non-speech segments of recorded speech. Taking an information-theoretic approach, we use Gaussian Mixture Model (GMM) trained with maximum mutual information (MMI) to represent device-specific features. Experimental results using Mel-frequency and linear frequency cepstral coefficients (MFCC and LFCC) show that features extracted from the non-speech segments of speech contain higher mutual information and yields higher recognition rates than those from speech portions or the whole utterance. Identification rate improves from 96.42 % to 98.39 % and equal error rate (EER) reduces from 1.20% to 0.47 % when non-speech parts are used to extract features. Recognition results are provided with classical GMM trained both with maximum likelihood (ML) and maximum mutual information (MMI) criteria, as well as support vector machines (SVMs). Identification under additive noise case is also considered and it is shown that identification rates reduces dramatically in case of additive noise.

show abstract

Section: Comparison Of Mfcc and Lfcc Featuresmentioning

confidence: 99%

Source cell-phone recognition from recorded speech using non-speech segments

Hanilçi

Kinnunen

2014

Digital Signal Processing

View full text Add to dashboard Cite

show abstract

“…Statistical significance of these comparisons are represented with scores. Note that these results should be analyzed using Table 4 (given in [27,31]) which presents the confidence levels as an indicator of the statistical significance based on scores. In terms of statistical significance, the binary case resulted in significant results for 3 out of 8 datasets, while the floating-point descriptor yielded statistically significant results for the complete dataset.…”

Section: Resultsmentioning

confidence: 99%

“…The test creates 2 × 2 contingency tables in order to compute scores. This test has a published past of usage by medical research community [30] and has recently been used for performance comparison in computer vision for the first time by Clark [7] and later for machine learning by Bostanci [27]. The test is significantly robust against Type-I error, i.e.…”

Section: Discussionmentioning

confidence: 99%

“…A GA was used here to improve the robustness of the image descriptors under severe cases of each condition. A detailed statistical analysis using Mc Nemar's test [27] showed that there are statistically significant differences between the performances of the original and the evolved descriptors. The rest of the paper is structured as follows: Section 2 gives details of the evolutionary approach for descriptors with binary and floating-point structure.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Resilient Image Feature Description through Evolution

Bostancı

2017

ijisae

Self Cite

View full text Add to dashboard Cite

Feature description is an important stage in many different vision algorithms. Image features detected by various detectors can be described using descriptors either with a binary or floating-point structure. This study presents the use of evolutionary algorithms, namely Genetic Algorithms (GA), in order to improve the robustness of the feature descriptors against increasing levels of photographic distortions such as noise or JPEG compression. Original feature descriptors were evolved in order to reduce the descriptor distance for the mentioned test cases. Results, tested using a statistical framework, suggest that the evolved descriptors offer better matching performance for two state-of-the-art descriptors.

show abstract

“…Room mean squared error (RMSE) indicates how precise the classification is. Lower RMSE value indicates the more accurate classifier results [12]. RMSE value is also used in this research for evaluation.…”

Section: B Evaluation Criterionmentioning

confidence: 99%

Analysis of the Effect of Clustering the Training Data in Naive Bayes Classifier for Anomaly Network Intrusion Detection

Subramanian¹,

Ong²

2014

JACN

View full text Add to dashboard Cite

Abstract-This paper presents the analysis of the effect of clustering the training data and test data in classification efficiency of Naive Bayes classifier. KDD cup 99 benchmark dataset is used in this research. The training set is clustered using k means clustering algorithm into 5 clusters. Then 8800 samples are taken from the clusters to form the training and test set. The results are compared with that of two Naive Bayes classifiers trained on random sampled data containing 8800 and 17600 instances respectively. The main contribution of this paper is that it is empirically proved that the training set derived from clusters generated by k-means clustering algorithm improves the classification efficiency of the Naive Bayes classifier. The results show the accuracy of the Naive Bayes classifier trained with clustered instances is 94.4% while that of normal instances are 85.41% and 89.26%.

show abstract

An Evaluation of Classification Algorithms Using Mc Nemar’s Test

Cited by 53 publications

References 10 publications

Source cell-phone recognition from recorded speech using non-speech segments

Source cell-phone recognition from recorded speech using non-speech segments

Resilient Image Feature Description through Evolution

Analysis of the Effect of Clustering the Training Data in Naive Bayes Classifier for Anomaly Network Intrusion Detection

Contact Info

Product

Resources

About