Integrating complex valued hyperdimensional computing with modular artificial neural networks

Abstract

Traditional approaches using Deep Neural Networks for classification, while unquestionably successful, struggle with more general intelligence tasks such as "on the fly" learning as demonstrated by biological systems. Organisms possess myriad sensory organs for interacting with their environment. By the time these diverse sensory signals reach the brain; however, they are all converted into a spiking information representation, over which the brain itself operates. In a similar manner, myriad machine learning (ML) algorithms today compute on equally diverse data modalities; but without a consistent information representation for their respective outputs, these algorithms are frequently used independently of each other. Consequently, there is growing interest in information representations to unify these algorithms, with the larger goal of designing ML modules that may be arbitrarily arranged to solve larger-scale ML problems, analogous to digital circuit design today.One promising information representation is that of a "symbol" expressed as a high-dimensional vector, thousands of elements long. Hyperdimensional computing (HDC) is an algebra for the creation, manipulation, and measurement of correlations among "symbols" expressed as hypervectors. Towards this goal, an external plexiform layer (EPL) network, echo state network (ESN), and modern Hopfield network were adapted to implement the mathematical operations of complex phasor based HDC. Further, since symbol error correction is an important consideration for computing with networks of ML modules, a task agnostic minimum query similarity for complete symbol error correction was measured as a function of hypervector length. Based on these results, problem-independent similarities have been established within which HDC equations should be designed. Lastly, these ANNs were tested against several tasks representative of online and "plug & play" ML among expeditionary robots. For all criteria considered, the modern Hopfield network was the most capable ANN evaluated for use with complex phasor based HDC, providing 100% symbol recovery in a single time step for nearly all parameter settings.