Understanding Hyperdimensional Computing for Parallel Single-Pass Learning

Abstract

Hyperdimensional computing (HDC) is an emerging learning paradigm that computes with high dimensional binary vectors. It is attractive because of its energy efficiency and low latency, especially on emerging hardware-but HDC suffers from low model accuracy, with little theoretical understanding of what limits its performance. We propose a new theoretical analysis of the limits of HDC via a consideration of what similarity matrices can be "expressed" by binary vectors, and we show how the limits of HDC can be approached using random Fourier features (RFF). We extend our analysis to the more general class of vector symbolic architectures (VSA), which compute with high-dimensional vectors (hypervectors) that are not necessarily binary. We propose a new class of VSAs, finite group VSAs, which surpass the limits of HDC. Using representation theory, we characterize which similarity matrices can be "expressed" by finite group VSA hypervectors, and we show how these VSAs can be constructed. Experimental results show that our RFF method and group VSA can both outperform the state-of-theart HDC model by up to 7.6% while maintaining hardware efficiency.

LeHDC: Learning-Based Hyperdimensional Computing Classifier

“…In general, an HDC classifier can use record-based or 𝑁 -gram-based encoders [3]. While our training approach applies to any encoding methods (including advanced ones [20] based on sophisticated feature extractions), 2 for a concrete case study, we adopt the commonly-used record-based encoding which, shown in Eq. 1, has higher accuracy than the 𝑁gram-based method for many applications [3].…”

“…Per the DAC'22 policy, we are not allowed to make significant non-editorial changes to papers once accepted. Thus, as Ref [20]. (arXiv date 2/10/2022) was not available or cited at the time of our DAC'22 submission on 11/22/2021, it will not be included in the final camera-ready version of this paper.…”