The Mechanism Analysis of the Accelerator for Support Vector Regression Based on Data Partition

Abstract

Support vector regression is an important algorithm in machine learning, and it is widely used in real life for its good performance, such as house price forecast, disease prediction, weather forecast, and so on. However, it cannot efficiently process large-scale data, because it has a high time complexity in the training process. Data partition as an important solution to solve the large-scale learning problem mainly focuses on the classification task, it trains the classifiers over the divided subsets produced by data partition and obtain the final classifier by combining those classifiers. Meanwhile, the most existing method rarely study the influence of data partition on the regressor performance, so that it is difficult to keep its generation ability. To solve this problem, we obtain the estimation of the difference in objective function before and after the data partition. Mini-Batch K-Means clustering is adopted to largely reduce this difference, and an improved algorithm is proposed. This proposed algorithm includes training stage and prediction stage. In training stag, it uses Mini-Batch K-Means clustering to divide the input space into some disjoint sub-regions of equal sample size, then it trains the regressor on each divided sub-region using support vector regression algorithm. In the prediction stage, the regressor merely offers the predicted label for the unlabeled instances that are in the same sub-region. Experiment results on real datasets illustrate that the proposed algorithm obtains the similar generation ability as the original algorithm, but it has less execution time than other acceleration algorithms.