Cross Validation Through Two-Dimensional Solution Surface for Cost-Sensitive SVM

Abstract

Model selection plays an important role in cost-sensitive SVM (CS-SVM). It has been proven that the global minimum cross validation (CV) error can be efficiently computed based on the solution path for one parameter learning problems. However, it is a challenge to obtain the global minimum CV error for CS-SVM based on one-dimensional solution path and traditional grid search, because CS-SVM is with two regularization parameters. In this paper, we propose a solution and error surfaces based CV approach (CV-SES). More specifically, we first compute a two-dimensional solution surface for CS-SVM based on a bi-parameter space partition algorithm, which can fit solutions of CS-SVM for all values of both regularization parameters. Then, we compute a two-dimensional validation error surface for each CV fold, which can fit validation errors of CS-SVM for all values of both regularization parameters. Finally, we obtain the CV error surface by superposing K validation error surfaces, which can find the global minimum CV error of CS-SVM. Experiments are conducted on seven datasets for cost sensitive learning and on four datasets for imbalanced learning. Experimental results not only show that our proposed CV-SES has a better generalization ability than CS-SVM with various hybrids between grid search and solution path methods, and than recent proposed cost-sensitive hinge loss SVM with three-dimensional grid search, but also show that CV-SES uses less running time.

A Novel Redundant Data Storage Algorithm Based on Minimum Spanning Tree and Quasi-randomized Matrix

“…In the future research, we may consider how to utilize cooperative communication mechanism among neighboring nodes to realize reliable and efficient data storage. Recently more and more machine learning algorithms such as SVM have been applied to the research of wireless sensor networks, but there are some limitations in traditional SVM algorithm, we are very happy to see there are a lot of innovations to improve traditional SVM and machine learning approaches [18][19][20][21]. Machine learning approach can be applied to our work in our future research.…”

Integrating synthetic minority oversampling and gradient boosting decision tree for bogie fault diagnosis in rail vehicles

“…32 SVM is one of the best working classifiers, due to its excellent generalization ability. 33 It reduces the claim on data scale and distribution by structural description of data distribution with margin concept. SVM showed state-of-the-art performance in realworld applications such as text categorization 34 and bogie fault detection.…”

A Cost-Sensitive Deep Belief Network for Imbalanced Classification

“…Bertoni et al [35] proposed a cost-sensitive neural network for semi-supervised learning in graphs. Cost-sensitive SVM (CS-SVM) [36] was discussed with model selection via global minimum cross validation error. Tan et al [37] proposed an evolutionary fuzzy ARTMAP (FAM) neural network using adaptive incremental learning method to overcome the stability-plasticity dilemma on stream imbalanced data.…”