OS-ELM-FPGA: An FPGA-Based Online Sequential Unsupervised Anomaly Detector

Tsukada, Mineto; Kondo, Masaaki; Matsutani, Hiroki

doi:10.1007/978-3-030-10549-5_41

Cited by 8 publications

(10 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, studies on that of OS-ELM have just started to be reported in the past few years. Tsukada et al provided a theoretical analysis for hardware implementations of OS-ELM to significantly reduce the computational cost [12]. Villora et al and Safaei et al proposed fast and efficient FPGA based implementations of OS-ELM and showed the possibility of on-device learning on embedded devices [32,33].…”

Section: Hardware Implementation Of Os-elmmentioning

confidence: 99%

A Neural Network-Based On-device Learning Anomaly Detector for Edge Devices

Tsukada

Kondo

Matsutani

2020

IEEE Trans. Comput.

Self Cite

View full text Add to dashboard Cite

Semi-supervised anomaly detection is an approach to identify anomalies by modeling the distribution of normal data. Nowadays, backpropagation neural networks (i.e., BP-NNs) based approaches have been drawing attention because of their high generalization performance for a wide range of real-world data. In a typical application, such BP-NN-based models are iteratively optimized in server machines with a large amount of data gathered from edge devices. However, there are two issues in this framework: (1) BP-NNs' iterative optimization approach often takes too long a time to follow time-series changes of the distribution of normal data (i.e., concept drift), and (2) data transfers between the server machines and the edge devices have a risk to cause data breaches. To address these issues, we propose an ON-device sequential Learning semi-supervised Anomaly Detector called ONLAD and its FPGA-based IP core called ONLAD Core so that various kinds of resource-limited edge devices can use our approach. Experimental results show that ONLAD has favorable anomaly detection capability especially in an environment which simulates concept drift. Evaluations of ONLAD Core confirm that it can perform training and prediction computations faster than BP-NN-based software implementations by x1.95 ∼ x4.51 and x2.29 ∼ x4.73, respectively. We also demonstrate that our on-board implementation which integrates ONLAD Core works at x6.3 ∼ x25.4 lower power consumption while training computations are continuously executed.

show abstract

Section: Hardware Implementation Of Os-elmmentioning

confidence: 99%

A Neural Network-Based On-device Learning Anomaly Detector for Edge Devices

Tsukada

Kondo

Matsutani

2020

IEEE Trans. Comput.

Self Cite

View full text Add to dashboard Cite

show abstract

“…To enable the retraining a model at resource-limited edge devices, in this paper we use a neural network based ondevice learning approach [1], [2] since it can sequentially train neural networks at resource-limited edge devices and also the neural networks typically have a high flexibility to address various nonlinear problems. Its low-cost hardware implementation is also introduced in [2].…”

Section: Introductionmentioning

confidence: 99%

“…In this paper, we use the on-device learning approach [1], [2] based on OS-ELM (Online Sequential Extreme Learning Machine) [3] and autoencoder [4]. Autoencoder is a type of neural network architecture which can be applied to unsupervised or semi-supervised anomaly detection, and OS-ELM is used to sequentially train neural networks at resourcelimited edge devices.…”

Section: Introductionmentioning

confidence: 99%

“…It is applied to the OS-ELM based on-device learning approach to construct the on-device federated learning. Please note that although in this paper the on-device federated learning is applied to anomaly detection tasks since the baseline on-device learning approach [1], [2] is designed for anomaly detection tasks, the proposed approach that employs the concept of E 2 LM is more general and can be applied to the other machine learning tasks. In the evaluations, we will demonstrate that the proposed on-device federated learning can produce a merged model by integrating trained results from multiple edge devices as accurately as traditional backpropagation based neural networks and a traditional federated learning approach with lower computation or communication cost.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An On-Device Federated Learning Approach for Cooperative Model Update Between Edge Devices

2021

Self Cite

View full text Add to dashboard Cite

Most edge AI focuses on prediction tasks on resource-limited edge devices while the training is done at server machines. However, retraining or customizing a model is required at edge devices as the model is becoming outdated due to environmental changes over time. To follow such a concept drift, a neural-network based on-device learning approach is recently proposed, so that edge devices train incoming data at runtime to update their model. In this case, since a training is done at distributed edge devices, the issue is that only a limited amount of training data can be used for each edge device. To address this issue, one approach is a cooperative learning or federated learning, where edge devices exchange their trained results and update their model by using those collected from the other devices. In this paper, as an on-device learning algorithm, we focus on OS-ELM (Online Sequential Extreme Learning Machine) to sequentially train a model based on recent samples and combine it with autoencoder for anomaly detection. We extend it for an on-device federated learning so that edge devices can exchange their trained results and update their model by using those collected from the other edge devices. This cooperative model update is oneshot while it can be repeatedly applied to synchronize their model. Our approach is evaluated with anomaly detection tasks generated from a driving dataset of cars, a human activity dataset, and MNIST dataset. The results demonstrate that the proposed on-device federated learning can produce a merged model by integrating trained results from multiple edge devices as accurately as traditional backpropagation based neural networks and a traditional federated learning approach with lower computation or communication cost.

show abstract

“…In 2016, [7] proposed an FPGA implementation of a real-time extreme learning machine. Another variant of ELM namely Online Sequential Extreme Learning Machine (OS-ELM) is also implemented using FPGA [8][9]. The features of FPGA such as reconfigurability, high parallelism and low power consumption makes it as the best candidate to accelerate the highly computational algorithm such as pseudo-inverse hence optimizing the overall performance of ELM [10] [11].…”

mentioning

confidence: 99%

Accelerating Extreme Learning Machine on FPGA by Hardware Implementation of Given Rotation-QRD

Tan

Ismail

Ooi

et al. 2019

IJIE

View full text Add to dashboard Cite

Currently, Extreme Learning Machine (ELM) is one of the research trends in the machine learning field due to its remarkable performances in terms of complexity and computational speed. However, the big data era and the limitations of general-purpose processor cause the increasing of interest in hardware implementation of ELM in order to reduce the computational time. Hence, this work presents the hardware-software co-design of ELM to improve the overall performances. In the co-design paradigm, one of the important components of ELM, namely Given Rotation-QRD (GR-QRD) is developed as a hardware core. Field Programmable Gate Array (FPGA) is chosen as the platform for ELM implementation due to its reconfigurable capability and high parallelism. Moreover, the learning accuracy and computational time would be used to evaluate the performances of the proposed ELM design. Our experiment has shown that GR-QRD accelerator helps to reduce the computational time of ELM training by 41.75% while maintaining the same training accuracy in comparison to pure software of ELM.

show abstract

OS-ELM-FPGA: An FPGA-Based Online Sequential Unsupervised Anomaly Detector

Cited by 8 publications

References 11 publications

A Neural Network-Based On-device Learning Anomaly Detector for Edge Devices

A Neural Network-Based On-device Learning Anomaly Detector for Edge Devices

An On-Device Federated Learning Approach for Cooperative Model Update Between Edge Devices

Accelerating Extreme Learning Machine on FPGA by Hardware Implementation of Given Rotation-QRD

Contact Info

Product

Resources

About