Machine Learning for Microcontroller-Class Hardware: A Review

Saha, Swapnil Sayan; Sandha, Sandeep Singh; Srivastava, Mani

doi:10.1109/jsen.2022.3210773

Cited by 96 publications

(27 citation statements)

References 176 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, if we train our network with new data, after some iterations, the network will forget what it learned previously (catastrophic forgetting). So we need other methods to prevent this (Saha et al, 2022 ).…”

Section: Discussionmentioning

confidence: 99%

E-prop on SpiNNaker 2: Exploring online learning in spiking RNNs on neuromorphic hardware

et al. 2022

View full text Add to dashboard Cite

IntroductionIn recent years, the application of deep learning models at the edge has gained attention. Typically, artificial neural networks (ANNs) are trained on graphics processing units (GPUs) and optimized for efficient execution on edge devices. Training ANNs directly at the edge is the next step with many applications such as the adaptation of models to specific situations like changes in environmental settings or optimization for individuals, e.g., optimization for speakers for speech processing. Also, local training can preserve privacy. Over the last few years, many algorithms have been developed to reduce memory footprint and computation.MethodsA specific challenge to train recurrent neural networks (RNNs) for processing sequential data is the need for the Back Propagation Through Time (BPTT) algorithm to store the network state of all time steps. This limitation is resolved by the biologically-inspired E-prop approach for training Spiking Recurrent Neural Networks (SRNNs). We implement the E-prop algorithm on a prototype of the SpiNNaker 2 neuromorphic system. A parallelization strategy is developed to split and train networks on the ARM cores of SpiNNaker 2 to make efficient use of both memory and compute resources. We trained an SRNN from scratch on SpiNNaker 2 in real-time on the Google Speech Command dataset for keyword spotting.ResultWe achieved an accuracy of 91.12% while requiring only 680 KB of memory for training the network with 25 K weights. Compared to other spiking neural networks with equal or better accuracy, our work is significantly more memory-efficient.DiscussionIn addition, we performed a memory and time profiling of the E-prop algorithm. This is used on the one hand to discuss whether E-prop or BPTT is better suited for training a model at the edge and on the other hand to explore architecture modifications to SpiNNaker 2 to speed up online learning. Finally, energy estimations predict that the SRNN can be trained on SpiNNaker2 with 12 times less energy than using a NVIDIA V100 GPU.

show abstract

Section: Discussionmentioning

confidence: 99%

E-prop on SpiNNaker 2: Exploring online learning in spiking RNNs on neuromorphic hardware

et al. 2022

View full text Add to dashboard Cite

show abstract

“…However, general-purpose microcontrollers have very limited resources which pose a challenge to deep neural networks. [16,17] To address this challenge, we extend the concept of smart MEMS sensors further by integrating a hardware accelerator for CNNs.…”

Section: Smart Mems Sensorsmentioning

confidence: 99%

Vehicle Surroundings Perception Using Micro‐electromechanical Systems Inertial Sensors

Lahr,

Loh,

Schwarz

et al. 2024

Advanced Intelligent Systems

View full text Add to dashboard Cite

Sensors enable vehicles to perceive their surroundings even while parked. Camera‐based systems for vehicle surroundings perception are already available as of today. However, camera‐based systems result in excessive power consumption and data collection. In this article, a study on always‐on surroundings perception of parked vehicles is done using micro‐electromechanical systems (MEMS) inertial sensors as an alternative to camera‐based systems. The measurements of the MEMS inertial sensors are used to detect and classify 17 different types of events. These events include accidents, vandalism, rain, as well as normal use. A convolutional neural network (CNN) is used to classify the events. The data used to train the CNN is recorded using a test vehicle in a laboratory setting and on a test track. The CNN is quantized for deployment on a CNN hardware accelerator which is packaged with the sensor as a system‐in‐package (SiP). The novel sensor system achieves a classification accuracy of 88% at a total system power dissipation of 27 μW.

show abstract

“…TinyML software suites [19], including the open-sourced TensorFlow Lite Micro [43], EdgeML [44] and CMSIS-NN [45], allow for deploying neural networks on MCUs and are mainly designed for ARM Cortex-M and as such less attractive for RISC-V based processors. Similarly, Wulfert et al [46] present an object detection method for resourcelimited systems, performing camera-based human detection directly on a small ESP32 MCUs.…”

Section: Related Workmentioning

confidence: 99%

“…Consequently, there is a growing demand to enable semantic image understanding on the edge using ultralow power and constrained hardware. This shift has led to a surge of interest in various research areas, including architecture search, quantization techniques, and advanced inference engines tailored for resource-constrained devices [19]- [22]. MCUs are now being equipped with novel open-source energy-efficient cores, such as RISC-V cores, parallel processing engines, dedicated hardware accelerators, and specialized co-processors aimed at enabling efficient execution of complex machine learning tasks [23], [24].…”

Section: Introductionmentioning

confidence: 99%

Flexible and Fully Quantized Lightweight TinyissimoYOLO for Ultra-Low-Power Edge Systems

Moosmann,

Müller,

Zimmerman

et al. 2024

IEEE Access

View full text Add to dashboard Cite

This paper deploys and explores variants of TinyissimoYOLO, a highly flexible and fully quantized ultra-lightweight object detection network designed for edge systems with a power envelope of a few milliwatts. With experimental measurements, we present a comprehensive characterization of the network's detection performance, exploring the impact of various parameters, including input resolution, number of object classes, and hidden layer adjustments. We deploy variants of TinyissimoYOLO on state-of-the-art ultra-low-power extreme edge platforms, presenting a detailed comparison on latency, energy efficiency, and their ability to efficiently parallelize the workload. In particular, the paper presents a comparison between a RISC-V-based parallel processor (GAP9 from GreenWaves Technologies) with and without use of its on-chip hardware accelerator, an ARM Cortex-M7 core (STM32H7 from ST Microelectronics), two ARM Cortex-M4 cores (STM32L4 from ST Microelectronics and Apollo4b from Ambiq), and a multi-core platform aimed at edge AI applications with a CNN hardware accelerator (MAX78000 from Analog Devices). Experimental results show that the GAP9's hardware accelerator achieves the lowest inference latency and energy at 2.12 ms and 150 µJ respectively, which is around 2x faster and 20% more energy efficient than the next best platform, the MAX78000. The hardware accelerator of GAP9 can even run an increased resolution version of TinyissimoYOLO with 112 × 112 pixels and 10 detection classes within 3.2 ms, consuming 245 µJ. We also deployed and profiled a multi-core implementation on GAP9 at different core voltages and frequencies, achieving 11.3 ms with the lowest-latency and 490 µJ with the most energy-efficient configuration. With this paper, we demonstrate the flexibility of TinyissimoYOLO and prove its detection accuracy on a widely used detection dataset. Furthermore, we demonstrate its suitability for real-time ultra-low-power edge inference.

show abstract

Machine Learning for Microcontroller-Class Hardware: A Review

Cited by 96 publications

References 176 publications

E-prop on SpiNNaker 2: Exploring online learning in spiking RNNs on neuromorphic hardware

E-prop on SpiNNaker 2: Exploring online learning in spiking RNNs on neuromorphic hardware

Vehicle Surroundings Perception Using Micro‐electromechanical Systems Inertial Sensors

Flexible and Fully Quantized Lightweight TinyissimoYOLO for Ultra-Low-Power Edge Systems

Contact Info

Product

Resources

About