Increasing the Inference and Learning Speed of Tsetlin Machines with Clause Indexing

Gorji, Saeed Rahimi; Granmo, Ole‐Christoffer; Glimsdal, Sondre; Edwards, Jonathan; Goodwin, Morten

doi:10.1007/978-3-030-55789-8_60

Cited by 6 publications

(2 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…By introducing real-valued clause weights, the number of clauses can be reduced by up to 50× without accuracy loss [14]. Also, the logical inference structure of TMs makes it possible to index the clauses on the features that falsify them, increasing inference and learning speed by up to an order of magnitude [15]. In [16], stochastic searching on the line automata [17] learns integer clause weights, performing on-par or better than Random Forest, Gradient Boosting, Neural Additive Models, StructureBoost and Explainable Boosting Machines.…”

Section: Introductionmentioning

confidence: 99%

On the Convergence of Tsetlin Machines for the XOR Operator

Jiao

Zhang

Granmo

et al. 2023

IEEE Trans. Pattern Anal. Mach. Intell.

Self Cite

View full text Add to dashboard Cite

The Tsetlin Machine (TM) is a novel machine learning algorithm with several distinct properties, including transparent inference and learning using hardware-near building blocks. Although numerous papers explore the TM empirically, many of its properties have not yet been analyzed mathematically. In this article, we analyze the convergence of the TM when input is non-linearly related to output by the XOR-operator. Our analysis reveals that the TM, with just two conjunctive clauses, can converge almost surely to reproducing XOR, learning from training data over an infinite time horizon. Furthermore, the analysis shows how the hyper-parameter T guides clause construction so that the clauses capture the distinct sub-patterns in the data. Our analysis of convergence for XOR thus lays the foundation for analyzing other more complex logical expressions. These analyses altogether, from a mathematical perspective, provide new insights on why TMs have obtained the state-of-the-art performance on several pattern recognition problems.

show abstract

Section: Introductionmentioning

confidence: 99%

On the Convergence of Tsetlin Machines for the XOR Operator

Jiao

Zhang

Granmo

et al. 2023

IEEE Trans. Pattern Anal. Mach. Intell.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Further, hyper‐parameter search can be simplified with multi‐granular clauses, eliminating the pattern specificity parameter (Gorji et al, 2019). By indexing the clauses on the features that falsify them, up to an order of magnitude faster inference and learning has been reported (Gorji et al, 2020). Additionally, regression TMs compare favourably with Regression Trees, Random Forest Regression and Support Vector Regression (Abeyrathna, Granmo, Zhang, & Goodwin, 2019).…”

Section: Introductionmentioning

confidence: 99%

A multi‐step finite‐state automaton for arbitrarily deterministic Tsetlin Machine learning

et al. 2021

View full text Add to dashboard Cite

Due to the high arithmetic complexity and scalability challenges of deep learning, there is a critical need to shift research focus towards energy efficiency. Tsetlin Machines (TMs) are a recent approach to machine learning (ML) that has demonstrated significantly reduced energy compared to neural networks alike, while providing comparable accuracy on several benchmarks. However, TMs rely heavily on energy‐costly random number generation to stochastically guide a team of Tsetlin Automata (TA) in TM learning. In this paper, we propose a novel finite‐state learning automaton that can replace the TA in the TM, for increased determinism. The new automaton uses multi‐step deterministic state jumps to reinforce sub‐patterns, without resorting to randomization. A determinism parameter d finely controls trading off the energy consumption of random number generation, against randomization for increased accuracy. Randomization is controlled by flipping a coin before every d'th state jump, ignoring the state jump on tails. For example, d=1 makes every update random and d=∞ makes the automaton completely deterministic. Both theoretically and empirically, we establish that the proposed automaton converges to the optimal action almost surely. Further, used together with the TM, only substantial degrees of determinism reduce accuracy. Energy‐wise, random number generation constitutes switching energy consumption of the TM, saving up to 11 mW power for larger datasets with high d values. Our new learning automaton approach thus facilitates low‐energy ML.

show abstract