Multivariate Stochastic Approximation to Tune Neural Network Hyperparameters for Criticial Infrastructure Communication Device Identification

Bihl, Trevor J.; Steeneck, Daniel W.

doi:10.24251/hicss.2018.279

Cited by 4 publications

(22 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…a coder experientially finding settings that "just work," or 3) random searches which use random seeds (notably a competitive method). Grid searches involve creating an experimental design where design points are explored and then one uses either a spreadsheet search or a response surface method to find suitable operating points [21].…”

Section: Ai Hyperparameter Determinationmentioning

confidence: 99%

“…• Stochastic Approximation [21], hill climbing where hyperparameters are individually and sequentially changed • Evolutionary algorithms [20], which randomly start, select the best initial results (parents), and then generate multiple possible outcomes (children), and then repeat the process • Bayesian optimization (BO) [5] which treats the objective function as a random function and uses randomly determined hyperparameters to construct a distribution around the results • Other approaches which do not fit cleanly into these three groups, e.g. Radial Basis Functions [22], Hyberband [23], Nedler-Mead [24], and spectral approaches [25].…”

Section: Ai Hyperparameter Determinationmentioning

confidence: 99%

See 1 more Smart Citation

Easy and Efficient Hyperparameter Optimization to Address Some Artificial Intelligence “ilities”

Bihl¹,

Schoenbeck²,

Steeneck³

et al. 2020

Proceedings of the Annual Hawaii International Conference on System Sciences

Self Cite

View full text Add to dashboard Cite

Artificial Intelligence (AI), has many benefits, including the ability to find complex patterns, automation, and meaning making. Through these benefits, AI has revolutionized image processing among numerous other disciplines. AI further has the potential to revolutionize other domains; however, this will not happen until we can address the "ilities": repeatability, explain-ability, reliability, use-ability, trust-ability, etc. Notably, many problems with the "ilities" are due to the artistic nature of AI algorithm development, especially hyperparameter determination. AI algorithms are often crafted products with the hyperparameters learned experientially. As such, when applying the same algorithm to new problems, the algorithm may not perform due to inappropriate settings. This research aims to provide a straightforward and reliable approach to automatically determining suitable hyperparameter settings when given an AI algorithm. Results, show reasonable performance is possible and end-to-end examples are given for three deep learning algorithms and three different data problems.

show abstract

Section: Ai Hyperparameter Determinationmentioning

confidence: 99%

Section: Ai Hyperparameter Determinationmentioning

confidence: 99%

Easy and Efficient Hyperparameter Optimization to Address Some Artificial Intelligence “ilities”

Bihl¹,

Schoenbeck²,

Steeneck³

et al. 2020

Proceedings of the Annual Hawaii International Conference on System Sciences

Self Cite

View full text Add to dashboard Cite

show abstract

“…Notably, GRLVQI and machine learning algorithms in general are highly sensitive to hyperparameter settings, such as learning rates and architecture size [28]. Although work has considered finding optimal settings for such algorithms, e.g., GRLVQI-SD, GRLVQI with optimized hyperparameters for Stochastic Optimization via Sequential Design of [28], such approaches are computationally costly with dozens of iterations needed to obtain improved algorithm settings. Additionally, such highly tuned hyperparameter values are often specific to the scope of the data and thus not useable on other datasets.…”

Section: Classification Algorithms For Rf Fingerprintingmentioning

confidence: 99%

“…Classifier models were developed for the Z-wave dataset using four classifiers: (1) the proposed Cosine GRLVQI (Section 3.2), (2) MDA, (3) the baseline GRLVQI of Harmer et al [6], and (4) the GRLVQI-SD of Bihl and Steeneck [28]. Consistent with [24,62], for these classifiers, the following process of Algorithm 1 was employed for each SNR.…”

Section: Classifier Algorithm Performancementioning

confidence: 99%

See 1 more Smart Citation

Cyber-Physical Security with RF Fingerprint Classification through Distance Measure Extensions of Generalized Relevance Learning Vector Quantization

Bihl

Paciencia

Bauer

et al. 2020

Security and Communication Networks

View full text Add to dashboard Cite

Radio frequency (RF) fingerprinting extracts fingerprint features from RF signals to protect against masquerade attacks by enabling reliable authentication of communication devices at the “serial number” level. Facilitating the reliable authentication of communication devices are machine learning (ML) algorithms which find meaningful statistical differences between measured data. The Generalized Relevance Learning Vector Quantization-Improved (GRLVQI) classifier is one ML algorithm which has shown efficacy for RF fingerprinting device discrimination. GRLVQI extends the Learning Vector Quantization (LVQ) family of “winner take all” classifiers that develop prototype vectors (PVs) which represent data. In LVQ algorithms, distances are computed between exemplars and PVs, and PVs are iteratively moved to accurately represent the data. GRLVQI extends LVQ with a sigmoidal cost function, relevance learning, and PV update logic improvements. However, both LVQ and GRLVQI are limited due to a reliance on squared Euclidean distance measures and a seemingly complex algorithm structure if changes are made to the underlying distance measure. Herein, the authors (1) develop GRLVQI-D (distance), an extension of GRLVQI to consider alternative distance measures and (2) present the Cosine GRLVQI classifier using this framework. To evaluate this framework, the authors consider experimentally collected Z-wave RF signals and develop RF fingerprints to identify devices. Z-wave devices are low-cost, low-power communication technologies seen increasingly in critical infrastructure. Both classification and verification, claimed identity, and performance comparisons are made with the new Cosine GRLVQI algorithm. The results show more robust performance when using the Cosine GRLVQI algorithm when compared with four algorithms in the literature. Additionally, the methodology used to create Cosine GRLVQI is generalizable to alternative measures.

show abstract

Stochastic approximation for learning rate optimization for generalized relevance learning vector quantization

Steeneck

Bihl

2017

2017 IEEE National Aerospace and Electronics Conference (NAECON)

View full text Add to dashboard Cite

Multivariate Stochastic Approximation to Tune Neural Network Hyperparameters for Criticial Infrastructure Communication Device Identification

Abstract: The

Cited by 4 publications

References 38 publications

Easy and Efficient Hyperparameter Optimization to Address Some Artificial Intelligence “ilities”

Easy and Efficient Hyperparameter Optimization to Address Some Artificial Intelligence “ilities”

Cyber-Physical Security with RF Fingerprint Classification through Distance Measure Extensions of Generalized Relevance Learning Vector Quantization

Stochastic approximation for learning rate optimization for generalized relevance learning vector quantization

Contact Info

Product

Resources

About