The Tsetlin Machine -- A Game Theoretic Bandit Driven Approach to Optimal Pattern Recognition with Propositional Logic

Abstract

Although simple individually, artificial neurons provide state-of-the-art performance when interconnected in deep networks. Unknown to many, there exists an arguably even simpler and more versatile learning mechanism, namely, the Tsetlin Automaton. Merely by means of a single integer as memory, it learns the optimal action in stochastic environments through increment and decrement operations. In this paper, we introduce the Tsetlin Machine, which solves complex pattern recognition problems with easy-to-interpret propositional formulas, composed by a collective of Tsetlin Automata. To eliminate the longstanding problem of vanishing signal-to-noise ratio, the Tsetlin Machine orchestrates the automata using a novel game. Our theoretical analysis establishes that the Nash equilibria of the game align with the propositional formulas that provide optimal pattern recognition accuracy. This translates to learning without local optima, only global ones. We argue that the Tsetlin Machine finds the propositional formula that provides optimal accuracy, with probability arbitrarily close to unity. In five benchmarks, the Tsetlin Machine provides competitive accuracy compared with SVMs, Decision Trees, Random Forests, Naive Bayes Classifier, Logistic Regression, and Neural Networks. The Tsetlin Machine further has an inherent computational advantage since both inputs, patterns, and outputs are expressed as bits, while recognition and learning rely on bit manipulation. The combination of accuracy, interpretability, and computational simplicity makes the Tsetlin Machine a promising tool for a wide range of domains. Being the first of its kind, we believe the Tsetlin Machine will kick-start new paths of research, with a potentially significant impact on the AI field and the applications of AI.

On the Convergence of Tsetlin Machines for the AND and the OR Operators

“…where T h is a predefined threshold for classification. Note that for voting architecture, the TM can assign a polarity to each TA team [1]. For example, TA teams with odd indexes possess positive polarity, and they vote for class i.…”

“…Table 1: Type I Feedback -Feedback upon receiving a sample with label y = 1, for a single TA to decide whether to Include or Exclude a given literal x k /¬x k into C i j . NA means not applicable [1].…”

“…If the output from the training sample is y i = 1, we generate, in probability, Type I Feedback for each clause C i j ∈ C i , where C i is the set of clauses that are trained for pattern i. The probability of generating Type I Feedback is [1]:…”

“…The Tsetlin Machine (TM) [1] organizes groups of Tsetlin Automata (TAs) [2] to collaboratively learn distinct patterns in training data. A TA, which is the core learning entity of TM, is a kind of learning automata that select their current actions based on past experiences learnt from the environment in order to obtain the maximum reward.…”

“…TMs possess two main advantages: transparent inference and hardware friendliness [1,5]. TMs provide transparent learning that shows how sub-patterns are composed in clauses, by including or excluding certain literals based on the feedback that they receive.…”