On the Convergence of Tsetlin Machines for the XOR Operator

Abstract

The Tsetlin Machine (TM) is a novel machine learning algorithm with several distinct properties, including transparent inference and learning using hardware-near building blocks. Although numerous papers explore the TM empirically, many of its properties have not yet been analyzed mathematically. In this article, we analyze the convergence of the TM when input is non-linearly related to output by the XOR-operator. Our analysis reveals that the TM, with just two conjunctive clauses, can converge almost surely to reproducing XOR, learning from training data over an infinite time horizon. Furthermore, the analysis shows how the hyper-parameter T guides clause construction so that the clauses capture the distinct sub-patterns in the data. Our analysis of convergence for XOR thus lays the foundation for analyzing other more complex logical expressions. These analyses altogether, from a mathematical perspective, provide new insights on why TMs have obtained the state-of-the-art performance on several pattern recognition problems.

VDHLA: Variable Depth Hybrid Learning Automaton and Its Application to Defense Against the Selfish Mining Attack in Bitcoin

“…The experimented SVDHLAs have 1, 4, 6 initial depth and simultaneously, AVDHLAs have (1, 1), (4, 4), (6,6) initial depth. Fig 10 shows the result from the TNR and TNAS aspects.…”

“…Afterward, learning automaton's field rose to fame. A wide range of learning automaton's application has been reported since the last decade including: 1-Pattern Recognition: Convergence of Tsetlin machine for XOR operator [6], identity and not operator [7] 2-Neural Networks: Convolutional Regression [8], The similarity between perceptrons and Tsetlin [9], Deep neural network [10] 3-NLP: Solving pattern recognition tasks using propositional logic [11], Semantic representation of words [12] 4-Optimization problem: Particle swarm [13], multilevel optimization [14] 5-Graph theory: Partitioning problem [15], [16] 6-Computer Networks: Cognitive radio [18], Load balancing [19] and Wireless [17].…”

REDRESS: Generating Compressed Models for Edge Inference Using Tsetlin Machines

“…Theoretical proof of TM's capability to solve complex pattern recognition problems and derivations of propositional formulas and its alignment with Nash equilibrium can be found in [18]. Interested readers can find the proof of convergence of TM in [25], [26] and further details on theoretical aspects of TM in [27], [28], [29], [30]. In this section, we visualize the details of the working principle and actual implementation of TM algorithm.…”

A multi‐step finite‐state automaton for arbitrarily deterministic Tsetlin Machine learning

“…Additionally, the Markov property of TM learning has been utilized previously to analyse learning convergence (Jiao et al, 2021; Zhang et al, 2020). Because TM learning can be formulated as a Markov chain, we can mathematically prove convergence properties.…”

“…Finally, TMs have recently been shown to be fault‐tolerant, completely masking stuck‐at faults (Shafik et al, 2020). The convergence property of TM has recently been studied in (Jiao et al, 2021; Zhang et al, 2020).…”