A Hardware-Efficient ADMM-Based SVM Training Algorithm for Edge Computing

Abstract

This work demonstrates a hardware-efficient support vector machine (SVM) training algorithm via the alternative direction method of multipliers (ADMM) optimizer. Low-rank approximation is exploited to reduce the dimension of the kernel matrix by employing the Nyström method. Verified in four datasets, the proposed ADMM-based training algorithm with rank approximation reduces 32× of matrix dimension with only 2% drop in inference accuracy. Compared to the conventional sequential minimal optimization (SMO) algorithm, the ADMMbased training algorithm is able to achieve a 9.8×10 7 shorter latency for training 2048 samples. Hardware design techniques, including pre-computation and memory sharing, are proposed to reduce the computational complexity by 62% and the memory usage by 60%. As a proof of concept, an epileptic seizure detector chip is designed to demonstrate the effectiveness of the proposed hardware-efficient training algorithm. The chip achieves a 153,310× higher energy efficiency and a 364× higher throughput-to-area ratio for SVM training than a high-end CPU. This work provides a promising solution for edge devices which require low-power and real-time training.

Training very large scale nonlinear SVMs using Alternating Direction Method of Multipliers coupled with the Hierarchically Semi-Separable kernel approximations

“…Indeed, in this framework, approximating the kernel matrix with an HSS structure (h fixed) results in a highly efficient optimization phase for a fixed value of C (see Section 3.3). It is important to note, moreover, that the computational footprint related to the kernel matrix approximation phase is fully justified by the fact that the same approximation can be reused for training the model with different values of C; this feature makes our proposal particularly attractive when a fine grid is used for the tuning of the penalization parameter C. It is important to note, at this stage, that also the works [22,43] analyse the use of ADMM for SVMs: in [43] ADMM has been used to solve linear SVMs with feature selection whereas in [22] a hardware-efficient nonlinear SVM training algorithm has been presented in which the Nyström approximation is exploited to reduce the dimension of the kernel matrices. Both works represent and use, somehow, different frameworks and techniques from those presented here.…”

Edge Network Optimization Based on AI Techniques: A Survey

“…In [40], the authors proposed an alternative direction method of multipliers (ADMM) optimizer in order to reduce the dimensions of the Kernel matrix using the Nystrom technique. This ADMM algorithm reduces the dimensions by 32 times with a 2% decrease in accuracy.…”