Local Intrinsic Dimensionality of IoT Networks for Unsupervised Intrusion Detection

Gorbett, Matt; Shirazi, Hossein; Ray, Indrakshi

doi:10.1007/978-3-031-10684-2_9

Lecture Notes in Computer Science

2022

DOI: 10.1007/978-3-031-10684-2_9

|View full text |Cite

Local Intrinsic Dimensionality of IoT Networks for Unsupervised Intrusion Detection

Matt Gorbett

Hossein Shirazi

Indrakshi Ray

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...

Citation Types

Supporting

Mentioning

Contrasting

Year Published

2023

2024

Publication Types

Select...

Other2

Relationship

Self Cite0

Independent2

Authors

Journals

Cited by 2 publications

(1 citation statement)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In other words, Transformers are expressive and powerful enough for certain tasks that we can prune the models in an unsophisticated way and maintain accuracy. Moreover, many time series datasets and datasets generated at the edge are often times simplistic enough that we can apply this unsophisticated pruning [22,23].…”

mentioning

confidence: 99%

Sparse Binary Transformers for Multivariate Time Series Modeling

Gorbett

Shirazi

Ray

2023

Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

View full text Add to dashboard Cite

Compressed Neural Networks have the potential to enable deep learning across new applications and smaller computational environments. However, understanding the range of learning tasks in which such models can succeed is not well studied. In this work, we apply sparse and binary-weighted Transformers to multivariate time series problems, showing that the lightweight models achieve accuracy comparable to that of dense floating-point Transformers of the same structure. Our model achieves favorable results across three time series learning tasks: classification, anomaly detection, and single-step forecasting. Additionally, to reduce the computational complexity of the attention mechanism, we apply two modifications, which show little to no decline in model performance: 1) in the classification task, we apply a fixed mask to the query, key, and value activations, and 2) for forecasting and anomaly detection, which rely on predicting outputs at a single point in time, we propose an attention mask to allow computation only at the current time step. Together, each compression technique and attention modification substantially reduces the number of non-zero operations necessary in the Transformer. We measure the computational savings of our approach over a range of metrics including parameter count, bit size, and floating point operation (FLOPs) count, showing up to a 53× reduction in storage size and up to 10.5× reduction in FLOPs. CCS CONCEPTS• Computing methodologies → Neural networks; Supervised learning.

show abstract

mentioning

confidence: 99%

Sparse Binary Transformers for Multivariate Time Series Modeling

Gorbett

Shirazi

Ray

2023

Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

View full text Add to dashboard Cite

show abstract

Tiled Bit Networks: Sub-Bit Neural Network Compression Through Reuse of Learnable Binary Vectors

Gorbett,

Shirazi,

Ray

2024

Proceedings of the 33rd ACM International Conference on Information and Knowledge Management

View full text Add to dashboard Cite

scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.

Contact Info

customersupport@researchsolutions.com

10624 S. Eastern Ave., Ste. A-614

Henderson, NV 89052, USA

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Blog Terms and Conditions API Terms Privacy Policy Contact Cookie Preferences Do Not Sell or Share My Personal Information

Made with 💙 for researchers

Part of the Research Solutions Family.

Local Intrinsic Dimensionality of IoT Networks for Unsupervised Intrusion Detection

Cited by 2 publications

References 39 publications

Sparse Binary Transformers for Multivariate Time Series Modeling

Sparse Binary Transformers for Multivariate Time Series Modeling

Tiled Bit Networks: Sub-Bit Neural Network Compression Through Reuse of Learnable Binary Vectors

Contact Info

Product

Resources

About