Memristor Based Autoencoder for Unsupervised Real-Time Network Intrusion and Anomaly Detection

Alam, Md. Shahanur; Fernando, B. Rasitha; Jaoudi, Yassine; Yakopcic, Chris; Hasan, Raqibul; Taha, Tarek M.; Subramanyam, Guru

doi:10.1145/3354265.3354267

Cited by 11 publications

(3 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To overcome this constraint, researchers have turned to unsupervised learning techniques, such as anomaly detection algorithms based on autoencoders [6,7]. Moreover, studies have been conducted on unsupervised deep learning circuits using memristors to enable real-time anomaly detection with autoencoders on low-power devices [35].…”

Section: Related Work: Anomaly Detection With Autoencoders and Nanosc...mentioning

confidence: 99%

Quantized non-volatile nanomagnetic domain wall synapse based autoencoder for efficient unsupervised network anomaly detection

Alam,

Misba,

Atulasimha

2024

Neuromorph. Comput. Eng.

View full text Add to dashboard Cite

Anomaly detection in real-time using autoencoders implemented on edge devices is exceedingly challenging due to limited hardware, energy, and computational resources. We show that these limitations can be addressed by designing an autoencoder with low-resolution non-volatile memory-based synapses and employing an effective quantized neural network learning algorithm. We further propose nanoscale ferromagnetic racetracks with engineered notches hosting magnetic domain walls (DW) as exemplary non-volatile memory based autoencoder synapses, where limited state (5-state) synaptic weights are manipulated by spin orbit torque (SOT) current pulses to write different magnetoresistance states. The performance of anomaly detection of the proposed autoencoder model is evaluated on the NSL-KDD dataset. Limited resolution and DW device stochasticity aware training of the autoencoder is performed, which yields comparable anomaly detection performance to the autoencoder having floating-point precision weights. While the limited number of quantized states and the inherent stochastic nature of DW synaptic weights in nanoscale devices are typically known to negatively impact the performance, our hardware-aware training algorithm is shown to leverage these imperfect device characteristics to generate an improvement in anomaly detection accuracy (90.98%) compared to accuracy obtained with floating-point synaptic weights that are extremely memory intensive. Furthermore, our DW-based approach demonstrates a remarkable reduction of at least three orders of magnitude in weight updates during training compared to the floating-point approach, implying significant reduction in operation energy for our method. This work could stimulate the development of extremely energy efficient non-volatile multi-state synapse-based processors that can perform real-time training and inference on the edge with unsupervised data.

show abstract

Section: Related Work: Anomaly Detection With Autoencoders and Nanosc...mentioning

confidence: 99%

Quantized non-volatile nanomagnetic domain wall synapse based autoencoder for efficient unsupervised network anomaly detection

Alam,

Misba,

Atulasimha

2024

Neuromorph. Comput. Eng.

View full text Add to dashboard Cite

show abstract

“…We note that onsite learning is attractive in power constraint edge devices, where the learning itself needs to adapt and respond to a continuously evolving environment. Embedded medical systems [56], real time intrusion detection [57], and dialect specific speech recognition systems can be benefitted from such onsite learning. Ex-site learning can perform inference tasks in edge devices with energy efficient manner (given the training is performed over cloud server), however the benefit can only apply to non-adaptive tasks.…”

Section: ) Ex-situ Trainingmentioning

confidence: 99%

Energy Efficient Learning With Low Resolution Stochastic Domain Wall Synapse for Deep Neural Networks

et al. 2022

View full text Add to dashboard Cite

We demonstrate extremely low resolution quantized (nominally 5-state) synapses with large stochastic variations in synaptic weight can be energy efficient and achieve reasonably high testing accuracies compared to Deep Neural Networks (DNNs) of similar sizes using floating precision synaptic weights. Specifically, voltage-controlled domain wall (DW) devices demonstrate stochastic behavior and can only encode limited states; however, they are extremely energy efficient during both training and inference. In this study, we propose both in-situ and ex-situ training algorithms, based on modification of the algorithm proposed by Hubara et al. [1] which works well with quantization of synaptic weights, and train several 5layer DNNs on MNIST dataset using 2-, 3-and 5-state DW devices as a synapse. For in-situ training, a separate high precision memory unit preserves and accumulates the weight gradients which prevents the accuracy loss due to weight quantization. For ex-situ training, a precursor DNN is first trained based on weight quantization and characterized DW device model. Moreover, a noise tolerance margin is included in both of the training methods to account for the intrinsic device noise. The highest inference accuracies we obtain after in-situ and ex-situ training are ~ 96.67% and ~96.63% which is very close to the baseline accuracy of ~97.1% obtained from a similar topology DNN having floating precision weights with no stochasticity. Large inter-state interval due to quantized weights and noise tolerance margin enables in-situ training with significantly lower number of programming attempts. Our proposed approach demonstrates a possibility of at least two orders of magnitude energy savings compared to the floating-point approach implemented in CMOS. This approach is specifically attractive for low power intelligent edge devices where the ex-situ learning can be utilized for energy efficient non-adaptive tasks and the in-situ learning can provide the opportunity to adapt and learn in a dynamically evolving environment.

show abstract

“…A memristive autoencoder computing system is designed for real-time intrusion detection and anomaly detection. The benchmark results show that the system has an overall detection accuracy of 92.91% and the malicious packet detection accuracy is 98.89% [100]. Autoencoder hardware is implemented by Cu:ZnO/Nb:STO memristor which can denoise sample imagines of the MNIST dataset [101].…”

Section: Autoencodermentioning

confidence: 99%

Memristive devices based hardware for unlabeled data processing

Xiao

Yan

Zhang

et al. 2022

Neuromorph. Comput. Eng.

View full text Add to dashboard Cite

Unlabeled data processing is of great significance for artificial intelligence (AI), since well-structured labelled data are scarce in a majority of practical applications due to the high cost of human annotation of labeling data. Therefore, automatous analysis of unlabeled datasets is important, and relevant algorithms for processing unlabeled data, such as k-means clustering, restricted Boltzmann machine and locally competitive algorithms etc., play a critical role in the development of AI techniques. Memristive devices offer potential for power and time efficient implementation of unlabeled data processing due to their unique properties in neuromorphic and in-memory computing. This review provides an overview of the design principles and applications of memristive devices for various unlabeled data processing and cognitive AI tasks.

show abstract

Memristor Based Autoencoder for Unsupervised Real-Time Network Intrusion and Anomaly Detection

Cited by 11 publications

References 23 publications

Quantized non-volatile nanomagnetic domain wall synapse based autoencoder for efficient unsupervised network anomaly detection

Quantized non-volatile nanomagnetic domain wall synapse based autoencoder for efficient unsupervised network anomaly detection

Energy Efficient Learning With Low Resolution Stochastic Domain Wall Synapse for Deep Neural Networks

Memristive devices based hardware for unlabeled data processing

Contact Info

Product

Resources

About