Scalable and fast SVM regression using modern hardware

Abstract

Support Vector Machine (SVM) regression is an important technique in data mining. The SVM training is expensive and its cost is dominated by: (i) the kernel value computation, and (ii) a search operation which finds extreme training data points for adjusting the regression function in every training iteration. Existing training algorithms for SVM regression are not scalable to large datasets because: (i) each training iteration repeatedly performs expensive kernel value computations, which is inefficient and requires holding the whole training dataset in memory; (ii) the search operation used in each training iteration considers the whole search space which is very expensive.In this article, we significantly improve the scalability and efficiency of SVM regression by exploiting the high performance of Graphics Processing Units (GPUs) and solid state drives (SSDs). Our key ideas are as follows. (i) To reduce the cost of repeated kernel value computations and avoid holding the whole training dataset in the GPU memory, we precompute all the kernel values and store them in the CPU memory extended by the SSD; together with an efficient strategy to read the precomputed kernel values, reusing precomputed kernel values with an efficient retrieval is much faster than computing them on-the-fly. This also alleviates the restriction that the training dataset has to fit into the GPU memory, and hence makes our algorithm scalable to large datasets, especially for large datasets with very high dimensionality. (ii) To enhance the performance of the frequently used search operation,

In Silico Prediction of Human Intravenous Pharmacokinetic Parameters with Improved Accuracy

“…9,11,34 On the basis of the structural risk minimization principle from statistical learning theory, SVM regression is one of the widely used machine learning methods in compound ADME properties and protein structural studies. 35,36 Random forest, first introduced by Stevens, 31 is an ensemble learning technique to improve the CART method using samples of the training data and random feature selection in tree induction. Gradient boosting machine is a family of powerful machine-learning techniques whose learning procedure consecutively fits new models to provide more a accurate estimate of the response variable.…”

“…Among a multitude of available modeling machine learning methods, several techniques including support vector machine (SVM), random forest (RF), gradient boosting machine (GBM), and XGBoost (XGB) were applied, which are highly effective, robust, and have been extensively used in QSAR modeling. ,, On the basis of the structural risk minimization principle from statistical learning theory, SVM regression is one of the widely used machine learning methods in compound ADME properties and protein structural studies. , Random forest, first introduced by Stevens, is an ensemble learning technique to improve the CART method using samples of the training data and random feature selection in tree induction. Gradient boosting machine is a family of powerful machine-learning techniques whose learning procedure consecutively fits new models to provide more a accurate estimate of the response variable.…”