Search for a substring of characters using the theory of non-deterministic finite automata and vector-character architecture

Pashchenko, Dmitry V.; Trokoz, Dmitry A.; Martyshkin, Alexey I.; Sinev, M.P; Svistunov, B. L.

doi:10.11591/eei.v9i3.1720

Cited by 11 publications

(8 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The best solution has a linear computational complexity, which depends on the sum of the lengths of the base and the query strings. Recently, an algorithm for substring search based on nondeterministic finitestate automata has been formulated with VSAs [Pashchenko et al, 2020]. We will briefly explain this formulation, and a modified version, because it nicely demonstrates how VSAs enable one to solve computer science problems when the framework is implemented with parallel hardware.…”

Section: Computing In Superposition With Vsas 1) Simple Examples Of C...mentioning

confidence: 99%

See 1 more Smart Citation

Vector Symbolic Architectures as a Computing Framework for Nanoscale Hardware

Kleyko¹,

Davies²,

Frady³

et al. 2021

Preprint

View full text Add to dashboard Cite

This article reviews recent progress in the development of the computing framework Vector Symbolic Architectures (also known as Hyperdimensional Computing). This framework is well suited for implementation in stochastic, nanoscale hardware and it naturally expresses the types of cognitive operations required for Artificial Intelligence (AI). We demonstrate in this article that the ring-like algebraic structure of Vector Symbolic Architectures offers simple but powerful operations on highdimensional vectors that can support all data structures and manipulations relevant in modern computing. In addition, we illustrate the distinguishing feature of Vector Symbolic Architectures, "computing in superposition," which sets it apart from conventional computing. This latter property opens the door to efficient solutions to the difficult combinatorial search problems inherent in AI applications. Vector Symbolic Architectures are Turing complete, as we show, and we see them acting as a framework for computing with distributed representations in myriad AI settings. This paper serves as a reference for computer architects by illustrating techniques and philosophy of VSAs for distributed computing and relevance to emerging computing hardware, such as neuromorphic computing.

show abstract

Section: Computing In Superposition With Vsas 1) Simple Examples Of C...mentioning

confidence: 99%

“…The reported values were averaged over 50 simulations. Left panel: the algorithm from [Pashchenko et al, 2020]; Right panel: the modified algorithm from [Pashchenko et al, 2020]. symbol of the query string q j .…”

Section: Computing In Superposition With Vsas 1) Simple Examples Of C...mentioning

confidence: 99%

Vector Symbolic Architectures as a Computing Framework for Nanoscale Hardware

Kleyko¹,

Davies²,

Frady³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…An algorithm for searching a query string in the base string was proposed in [Pashchenko et al, 2020] and modified in [Kleyko et al, 2021a]. It is built on top of the ideas for representing finite-state automata in an HV (see Section 2.1.1.1).…”

Section: String Processingmentioning

confidence: 99%

A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part II: Applications, Cognitive Models, and Challenges

Kleyko,

Rachkovskij,

Osipov

et al. 2021

Preprint

View full text Add to dashboard Cite

This is Part II of the two-part comprehensive survey devoted to a computing framework most commonly known under the names Hyperdimensional Computing and Vector Symbolic Architectures (HDC/VSA). Both names refer to a family of computational models that use high-dimensional distributed representations and rely on the algebraic properties of their key operations to incorporate the advantages of structured symbolic representations and vector distributed representations. Holographic Reduced Representations [Plate, 1995], [Plate, 2003] is an influential HDC/VSA model that is well-known in the machine learning domain and often used to refer to the whole family. However, for the sake of consistency, we use HDC/VSA to refer to the area.Part I of this survey [Kleyko et al., 2021c] covered foundational aspects of the area, such as historical context leading to the development of HDC/VSA, key elements of any HDC/VSA model, known HDC/VSA models, and transforming input data of various types into high-dimensional vectors suitable for HDC/VSA. This second part surveys existing applications, the role of HDC/VSA in cognitive computing and architectures, as well as directions for future work. Most of the applications lie within the machine learning/artificial intelligence domain, however we also cover other applications to provide a thorough picture. The survey is written to be useful for both newcomers and practitioners.

show abstract

“…In particular, here we consider the trajectory association task [22] as one of the ways of studying the short-term memory of a simplified version of the ESN [53]. The ability of being able to form and use short-term memory is a key enabler for many VSAs use-cases such as representation of data structures [54], [55], [56] and processing of strings [57], [58], [59].…”

Section: Appendix a Original Problem Formulationmentioning

confidence: 99%

Perceptron Theory for Predicting the Accuracy of Neural Networks

Kleyko,

Rosato,

Frady

et al. 2020

Preprint

View full text Add to dashboard Cite

Many neural network models have been successful at classification problems, but their operation is still treated as a black box. Here, we developed a theory for one-layer perceptrons that can predict performance on classification tasks. This theory is a generalization of an existing theory for predicting the performance of Echo State Networks and connectionist models for symbolic reasoning known as Vector Symbolic Architectures.In this paper, we first show that the proposed perceptron theory can predict the performance of types of Echo State Networks, which could not be described by the previous theory. Second, we apply our perceptron theory to the last layers of shallow randomly connected and deep multi-layer networks. The full theory is based on Gaussian statistics, but it is analytically intractable. We explore numerical methods to predict network performance for problems with a small number of classes. For problems with a large number of classes, we investigate stochastic sampling methods and a tractable approximation to the full theory. The quality of predictions is assessed in three experimental settings, using reservoir computing networks on a memorization task, shallow randomly connected networks on a collection of classification datasets, and deep convolutional networks with the ImageNet dataset.This study offers a surprisingly simple, bipartite approach to understand deep neural networks: the input is encoded by the last-but-one layers into a high-dimensional representation. This representation is mapped through the weights of the last layer into the postsynaptic sums of the output neurons. Specifically, the proposed perceptron theory uses the mean vector and covariance matrix of the postsynaptic sums to compute classification accuracies for the different classes. The first two moments of the distribution of the postsynaptic sums can predict the overall network performance quite accurately.

show abstract

Search for a substring of characters using the theory of non-deterministic finite automata and vector-character architecture

Cited by 11 publications

References 19 publications

Vector Symbolic Architectures as a Computing Framework for Nanoscale Hardware

Vector Symbolic Architectures as a Computing Framework for Nanoscale Hardware

A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part II: Applications, Cognitive Models, and Challenges

Perceptron Theory for Predicting the Accuracy of Neural Networks

Contact Info

Product

Resources

About