An area-efficient multiply-accumulation architecture and implementations for time-domain neural processing

Kawashima, Ichiro; Katori, Yuichi; Matsukawa, Takashi; Tamukoh, Hakaru

doi:10.1109/icfpt52863.2021.9609809

Cited by 7 publications

(1 citation statement)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Reservoir computing (RC) [12], [13], a type of recurrent neural network, presents an attractive prospect for edge AI applications, characterized by minimal training overhead. Notably, RC exhibits a potential for deployment on energy-efficient hardware platforms through specialized circuits [14]- [19], as well as leveraging physical processes for computation [20]- [25]. This distinct characteristic positions RC as an optimal candidate for edge AI implementations, aligning with considerations of power consumption.…”

Section: Introductionmentioning

confidence: 99%

Self-Organizing Multiple Readouts for Reservoir Computing

Tanaka,

Tamukoh

2023

IEEE Access

Self Cite

View full text Add to dashboard Cite

With advancements in deep learning (DL), artificial intelligence (AI) technology has become an indispensable tool. However, the application of DL incurs significant computational costs, making it less viable for edge AI scenarios. Consequently, the demand for cost-effective AI solutions, other than DL-based approaches, is increasing. Reservoir computing (RC) has attracted interest owing to its ability to provide lowcost training alternatives, holding great promise for edge AI applications. However, the training capability of RC is constrained by its reliance on a single linear layer, while weight connections in the remaining layers remain static during training. Moreover, accomplishing continuous learning tasks is difficult owing to the catastrophic forgetting in the linear layer. Therefore, we propose the integration of self-organizing multiple readouts to enhance RC's training capability. Our method distributes training data across multiple readouts, which prevents catastrophic forgetting of readouts and empowers each readout to adeptly assimilate new data, thereby elevating the overall training performance. The self-organizing function, which assigns similar data to the same readout, optimizes the memory utilization of these multiple readouts. Experimental results show that an RC equipped with the proposed multiple readouts successfully solved a continuous learning task by mitigating catastrophic forgetting because of the data distribution to the multiple readouts. Additionally, the RC achieved higher accuracy in a sound recognition task compared with the existing RC paradigm because of ensemble learning in the multiple readouts. Multiple readouts are effective in enhancing the training capability of RC and can contribute to the realization of RC applications.

show abstract

Section: Introductionmentioning

confidence: 99%

Self-Organizing Multiple Readouts for Reservoir Computing

Tanaka,

Tamukoh

2023

IEEE Access

Self Cite

View full text Add to dashboard Cite

show abstract

LUTNet-RC: Look-Up Tables Networks for Reservoir Computing on an FPGA

Yoshioka,

Tanaka,

Tamukoh

2023

2023 International Conference on Field Programmable Technology (ICFPT)

View full text Add to dashboard Cite

We propose look-up tables networks-based reservoir computing (LUTNet-RC). This work is the first trial of applying LUTNets to RC. LUTNet-RC consists of a LUT-based reservoir layer and a non-LUT-based output layer. LUTNets have disadvantages such as limited sparse connectivity and weights cannot be changed after implementation. However, when applied to a reservoir layer of RC (LUT-based reservoir layer), these disadvantages are eliminated, because this layer works with sparse connectivity and the weights are fixed, so only the advantage of small circuit resources is obtained. For the LUT-based reservoir layer, we propose and model a multi-bit weight reservoir, modifying the conventional binarized reservoir to improve calculation accuracy. In the case of LUTNets, the proposed multi-bit weight reservoir can be implemented without the increase in utilized circuit resources because LUTNets focus only on the input-output relationship on neurons. Additionally, we propose a speed-up method in the output layer with time division calculation, which compares the current network state with previous states and then calculates only status-changed neurons. As a result, we implement a LUTNet-RC with 1500 reservoir neurons on a field-programmable gate array (KR260) running at 100MHz. The utilized circuit resources are dominated by LUTs, which use approximately 26% of the total amount of LUTs. The LUTNet-RC can infer more than 10 6 data per second. We also verify the LUTNet-RC performance using nonlinear auto-regressive moving average 10 (NARMA10) and the performance is comparable to conventional works. We conclude that the LUTNet-RC is one of the highest-performance RC on an FPGA.Index Terms-neural networks, reservoir computing, field programmable gate array This work focuses on look-up tables networks (LUTNets)[7], one of the binarized neural networks (BNNs) [8], which is more specialized for FPGA implementation than general BNNs. LUTNets are neural networks constructed using neurons oriented to LUT elements on FPGAs, focusing only on the input-output relationship of neurons. LUTNets can operate with smaller circuit resources and higher recognition accuracy than general BNNs implemented on FPGAs [9] [10].LUTNets have disadvantages such as limited sparse connectivity, and weights cannot be changed after implementation. However, when applied to a reservoir layer of RC, these disadvantages are eliminated because this layer works with sparse connectivity, and the weights are fixed; thus, only the advantage of small circuit resources is obtained.From this good complementarity between RC and LUT-Nets, we propose LUTNet-based RC (LUTNet-RC), which consists of a LUT-based reservoir layer and a non-LUT-based output layer. We implemented LUTNet-RC on an FPGA and evaluated its computational accuracy and speed and circuit resources. The main contributions of this work are summarized as follows.• We discovered the good complementarity between RC and LUTNet and proposed LUTNet-RC, which is an application of LUTNet to RC. • We proposed a novel...

show abstract

Traffic Flow Optimization using a Chaotic Boltzmann Machine Annealer on an FPGA

Yoshioka,

Tanaka,

Tamukoh

2023

2023 International Conference on Field Programmable Technology (ICFPT)

View full text Add to dashboard Cite

We demonstrate a chaotic Boltzmann machine annealer, which is a field-programmable gate array (FPGA)-based high-performance annealing machine, and a control system powered by two open-source software packages. The annealer is employed in solving a large-scale real-world optimization problem, the traffic flow optimization. This involves distributing 500 car routes between the Haneda Airport area to the Pacifico Yokohama area, the venue of the International Conference on Field Programmable Technology 2023, while minimizing the total driving distance and preventing traffic jams. We obtain solutions that are comparable in accuracy to solutions of simulated annealing running on a graphics processing unit (GPU-SA) and a central processing unit (CPU-SA). The annealing machine on an FPGA is approximately 571 and 97600 times as fast as the GPU-SA and the CPU-SA, respectively. We will demonstrate solving traffic flow optimization in the towns familiar to the visitors.

show abstract

An area-efficient multiply-accumulation architecture and implementations for time-domain neural processing

Cited by 7 publications

References 7 publications

Self-Organizing Multiple Readouts for Reservoir Computing

Self-Organizing Multiple Readouts for Reservoir Computing

LUTNet-RC: Look-Up Tables Networks for Reservoir Computing on an FPGA

Traffic Flow Optimization using a Chaotic Boltzmann Machine Annealer on an FPGA

Contact Info

Product

Resources

About