Hypervector Design for Efficient Hyperdimensional Computing on Edge Devices

Basaklar, Toygun; Tuncel, Yigit; Narayana, Shruti Yadav; Gumussoy, Suat; Ogras, Umit Y.

doi:10.48550/arxiv.2103.06709

Cited by 3 publications

(4 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In contrast, HVs with a smaller dimensions may lose robustness but bring speed. Basaklar and co-workers 11 proposed to reduce dimensionality greatly from 8192 to 64 with little loss in accuracy. However, manual design of the HVs was required, which is obviously unsuitable for most alternate scenarios.…”

Section: Related Work and Motivationmentioning

confidence: 99%

Automated Architecture Search for Brain-inspired Hyperdimensional Computing

Yang¹,

Sangbong²,

Zhang³

et al. 2022

Preprint

View full text Add to dashboard Cite

This paper represents the first effort to explore an automated architecture search for hyperdimensional computing (HDC), a type of brain-inspired neural network. Currently, HDC design is largely carried out in an application-specific ad-hoc manner, which significantly limits its application. Furthermore, the approach leads to inferior accuracy and efficiency, which suggests that HDC cannot perform competitively against deep neural networks. Herein, we present a thorough study to formulate an HDC architecture search space. On top of the search space, we apply reinforcement-learning to automatically explore the HDC architectures. The searched HDC architectures show competitive performance on case studies involving a drug discovery dataset and a language recognition task. On the Clintox dataset, which tries to learn features from developed drugs that passed/failed clinical trials for toxicity reasons, the searched HDC architecture obtains the state-of-the-art ROC-AUC scores, which are 0.80% higher than the manually designed HDC and 9.75% higher than conventional neural networks. Similar results are achieved on the language recognition task, with 1.27% higher performance than conventional methods.

show abstract

Section: Related Work and Motivationmentioning

confidence: 99%

Automated Architecture Search for Brain-inspired Hyperdimensional Computing

Yang¹,

Sangbong²,

Zhang³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…To evaluate the existing HDC models, for fair comparison, we also implement the SOTA HDC classifier [15] with 𝐷 = 8, 000 using our acceleration designs. Moreover, we choose two other lightweight models for comparison: a compressed HDC model [2] that uses a small vector but has non-binary weights and per-feature ValueBoxes founded using evolutionary search; and SFC-fix with FINN [35] that applies a 3-layer binary MLP on FPGA. For these models, we only report their results available for our considered datasets.…”

Section: Hardware Accelerationmentioning

confidence: 99%

“…In addition, the ultra-high dimension is another fundamental drawback of HDC models. Simply reducing the dimension without fundamentally changing the HDC design can dramatically decrease the inference accuracy A recent study [2] reduces the dimension but considers non-binary HDC that is unfriendly to hardware acceleration. Moreover, it uses different sets of value vectors for different features, and hence results in a large model size (Table 3).…”

Section: Related Workmentioning

confidence: 99%

A Brain-Inspired Low-Dimensional Computing Classifier for Inference on Tiny Devices

Duan¹,

Xu²,

Ren³

2022

Preprint

View full text Add to dashboard Cite

By mimicking brain-like cognition and exploiting parallelism, hyperdimensional computing (HDC) classifiers have been emerging as a lightweight framework to achieve efficient on-device inference. Nonetheless, they have two fundamental drawbacks -heuristic training process and ultra-high dimension -which result in suboptimal inference accuracy and large model sizes beyond the capability of tiny devices with stringent resource constraints. In this paper, we address these fundamental drawbacks and propose a lowdimensional computing (LDC) alternative. Specifically, by mapping our LDC classifier into an equivalent neural network, we optimize our model using a principled training approach. Most importantly, we can improve the inference accuracy while successfully reducing the ultra-high dimension of existing HDC models by orders of magnitude (e.g., 8000 vs. 4/64). We run experiments to evaluate our LDC classifier by considering different datasets for inference on tiny devices, and also implement different models on an FPGA platform for acceleration. The results highlight that our LDC classifier offers an overwhelming advantage over the existing brain-inspired HDC models and is particularly suitable for inference on tiny devices.

show abstract

“…and images [15][16][17] can be represented using HDC. Current research findings have demonstrated that HDC can achieve comparable performance with traditional machine learning techniques but support few-shot learning [18][19][20][21][22][23], high energy efficiency [24][25][26][27][28][29][30][31][32][33][34], and hardware acceleration [35][36][37]. HDC has wide applications that are not limited to supervised learning (e.g., classification [38][39][40] and regression [41]), unsupervised learning (e.g., clustering [42][43][44][45]), and even reasoning [46][47][48].…”

Section: Introductionmentioning

confidence: 99%

Robust Clustering Using Hyperdimensional Computing

Ge,

Parhi

2024

IEEE Open J. Circuits Syst.

View full text Add to dashboard Cite

This paper addresses the clustering of data in the hyperdimensional computing (HDC) domain. In prior work, an HDC-based clustering framework, referred to as HDCluster, has been proposed. However, the performance of the existing HDCluster is not robust. The performance of HDCluster is degraded as the hypervectors for the clusters are chosen at random during the initialization step. To overcome this bottleneck, we assign the initial cluster hypervectors by exploring the similarity of the encoded data, referred to as query hypervectors. Intracluster hypervectors have a higher similarity than intercluster hypervectors. Harnessing the similarity results among query hypervectors, this paper proposes four HDCbased clustering algorithms: similarity-based k-means, equal bin-width histogram, equal bin-height histogram, and similarity-based affinity propagation. Experimental results illustrate that: (i) Compared to the existing HDCluster, our proposed HDC-based clustering algorithms can achieve better accuracy, more robust performance, fewer iterations, and less execution time. Similarity-based affinity propagation outperforms the other three HDC-based clustering algorithms on eight datasets by 2% ∼ 38% in clustering accuracy. (ii) Even for one-pass clustering, i.e., without any iterative update of the cluster hypervectors, our proposed algorithms can provide more robust clustering accuracy than HDCluster. (iii) Over eight datasets, five out of eight can achieve higher or comparable accuracy when projected onto the hyperdimensional space. Traditional clustering is more desirable than HDC when the number of clusters, k, is large.

show abstract

Hypervector Design for Efficient Hyperdimensional Computing on Edge Devices

Cited by 3 publications

References 24 publications

Automated Architecture Search for Brain-inspired Hyperdimensional Computing

Automated Architecture Search for Brain-inspired Hyperdimensional Computing

A Brain-Inspired Low-Dimensional Computing Classifier for Inference on Tiny Devices

Robust Clustering Using Hyperdimensional Computing

Contact Info

Product

Resources

About