AdaptHD: Adaptive Efficient Training for Brain-Inspired Hyperdimensional Computing

Abstract

Brain-inspired Hyperdimensional (HD) computing is a promising solution for energy-efficient classification. HD emulates cognition tasks by exploiting long-size vectors instead of working with numeric values used in contemporary processors. However, the existing HD computing algorithms have lack of controllability on the training iterations which often results in slow training or divergence. In this work, we propose AdaptHD, an adaptive learning approach based on HD computing to address the HD training issues. AdaptHD introduces the definition of learning rate in HD computing and proposes two approaches for adaptive training: iteration-dependent and data-dependent. In the iteration-dependent approach, AdaptHD uses a large learning rate to speedup the training procedure in the first iterations, and then adaptively reduces the learning rate depending on the slope of the error rate. In the data-dependent approach, AdaptHD changes the learning rate for each data point depending on how far off the data was misclassified. Our evaluations on a wide range of classification applications show that AdaptHD achieves 6.9× speedup and 6.3× energy efficiency improvement during training as compared to the state-of-the-art HD computing algorithm.

A Theoretical Perspective on Hyperdimensional Computing

“…A nice feature of this scheme is that it is extremely simple to implement in an on-line fashion: that is, on streaming data arriving continuously over time (Rahimi et al, 2018). It is common to fine-tune the class prototypes using a few rounds of perceptron training (Imani et al, , 2019b. Given some subsequent piece of query data x q ∈ X for which we do not know the correct label, we simply return the label of the most similar prototype:…”

Multi-Centroid Hyperdimensional Computing Approach for Epileptic Seizure Detection

“…Therefore, iterative learning tries to overcome the problem of single-pass learning that it can lead to the saturation of the prototype vectors of each class by data that are more common in each class and perform badly on under-represented patterns of the same class. In [12], authors tested iterative approaches with different fixed and adaptive learning rates on several datasets for speeding up learning and saving energy while keeping the same or higher accuracy as single-pass training.…”

“…Our proposed approach has a similar underlying idea as OnlineHD [23], or iterative learning [12] in that it focuses on less common patterns. However, it is different since it allows the creation of sub-classes rather than adding them multiple times to a single vector.…”

“…The initial 45 features, based on [18], contain 37 different entropy features, including sample, permutation, Renyi, Shannon, and Tsallis entropies, as well as 8 features from the frequency domain. For frequency-domain features, we compute the power spectral density and extract the relative power in the five common brain wave frequency bands; delta: [0.5-4] Hz, theta: [4][5][6][7][8] Hz, alpha: [8][9][10][11][12] Hz, beta: [12][13][14][15][16][17][18][19][20][21][22][23][24][25][26][27][28][29][30] Hz, gamma: [30-45] Hz, and a low-frequency component ([0-0.5] Hz), for each signal window. These features are commonly held to be medically relevant for detecting seizures [24].…”

“…An additional opportunity is the use of semi-supervised learning approaches with HD computing. In the literature, the form of iterative learning [12]) has been proposed, but it has not yet been fully explored for epilepsy. However, unsupervised or at least semi-supervised learning would be very useful due to the time-consuming and complex process of labeling data.…”