Robust and Computationally-Efficient Anomaly Detection Using Powers-Of-Two Networks

Muneeb, Usama; Koyuncu, Erdem; Keshtkarjahromi, Yasaman; Seferoğlu, Hülya; Erden, M.F.; Çetin, A. Enis

doi:10.1109/icassp40776.2020.9054442

Cited by 11 publications

(8 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Eicient neural network models include compressing a large neural network using quantization [22,23], hashing [24], pruning [25], vector quantization [26] and Hufman encoding [27]. Another approach is the SqueezeNet [28], which is designed as a small network with 1 × 1 convolutional ilters.…”

Section: Related Workmentioning

confidence: 99%

Block Walsh–Hadamard Transform-based Binary Layers in Deep Neural Networks

Pan¹,

Badawi²,

Çetin

2022

ACM Trans. Embed. Comput. Syst.

View full text Add to dashboard Cite

Convolution has been the core operation of modern deep neural networks. It is well-known that convolutions can be implemented in the Fourier Transform domain. In this paper, we propose to use binary block Walsh-Hadamard transform (WHT) instead of the Fourier transform. We use WHT-based binary layers to replace some of the regular convolution layers in deep neural networks. We utilize both one-dimensional (1-D) and two-dimensional (2-D) binary WHTs in this paper. In both 1-D and 2-D layers, we compute the binary WHT of the input feature map and denoise the WHT domain coefficients using a nonlinearity which is obtained by combining soft-thresholding with the tanh function. After denoising, we compute the inverse WHT. We use 1D-WHT to replace the 1 × 1 convolutional layers, and 2D-WHT layers can replace the 3 × 3 convolution layers and Squeeze-and-Excite layers. 2D-WHT layers with trainable weights can be also inserted before the Global Average Pooling (GAP) layers to assist the dense layers. In this way, we can reduce the number of trainable parameters significantly with a slight decrease in trainable parameters. In this paper, we implement the WHT layers into MobileNet-V2, MobileNet-V3-Large, and ResNet to reduce the number of parameters significantly with negligible accuracy loss. Moreover, according to our speed test, the 2D-FWHT layer runs about 24 times as fast as the regular 3 × 3 convolution with 19.51% less RAM usage in an NVIDIA Jetson Nano experiment.

show abstract

Section: Related Workmentioning

confidence: 99%

Block Walsh–Hadamard Transform-based Binary Layers in Deep Neural Networks

Pan¹,

Badawi²,

Çetin

2022

ACM Trans. Embed. Comput. Syst.

View full text Add to dashboard Cite

show abstract

“…Efficient neural network models include compressing a large neural network using quantization [22,23], hashing [24], pruning [25], vector quantization [26] and Huffman encoding [27]. Another approach is the SqueezeNet [28], which is designed as a small network with 1 × 1 convolutional filters.…”

Section: Related Workmentioning

confidence: 99%

Block Walsh-Hadamard Transform Based Binary Layers in Deep Neural Networks

Pan¹,

Badawi²,

Çetin³

2022

Preprint

View full text Add to dashboard Cite

Convolution has been the core operation of modern deep neural networks. It is well-known that convolutions can be implemented in the Fourier Transform domain. In this paper, we propose to use binary block Walsh-Hadamard transform (WHT) instead of the Fourier transform. We use WHT-based binary layers to replace some of the regular convolution layers in deep neural networks. We utilize both one-dimensional (1-D) and two-dimensional (2-D) binary WHTs in this paper. In both 1-D and 2-D layers, we compute the binary WHT of the input feature map and denoise the WHT domain coefficients using a nonlinearity which is obtained by combining soft-thresholding with the tanh function. After denoising, we compute the inverse WHT. We use 1D-WHT to replace the 1 × 1 convolutional layers, and 2D-WHT layers can replace the 3×3 convolution layers and Squeeze-and-Excite layers. 2D-WHT layers with trainable weights can be also inserted before the Global Average Pooling (GAP) layers to assist the dense layers. In this way, we can reduce the number of trainable parameters significantly with a slight decrease in trainable parameters. In this paper, we implement the WHT layers into MobileNet-V2, MobileNet-V3-Large, and ResNet to reduce the number of parameters significantly with negligible accuracy loss. Moreover, according to our speed test, the 2D-FWHT layer runs about 24 times as fast as the regular 3 × 3 convolution with 19.51% less RAM usage in an NVIDIA Jetson Nano experiment.

show abstract

“…Precision, F1-score, accuracy, recall, sensitivity, equal error rate (EER), specificity, and receiver operating curve (ROC) are also frequently used as metrics in this area. San Francisco cabspotting: [115] SBHAR: [46] SD-OCT: [17] Sentence polarity: [68] ShanghAaiTech: [41,75] SIXray: [91] Spectralis OCT: [26] SWaT system: [93] SVHN: [50,62] TalkingData AdTracking: [113,67] Tennessee eastman: [16,28] Texas coast: [27] Thyroid: [132] UBA: [96] UCI: [38,126] UCSD: [21,22,37,39,41,54,64,65,122,74,75,79,90,107,109] Udacity: [56,61] UMN: [39,43,64,65,74,90,107] UNSW-NB15: [110] VIRAT: [81] WADI test-bed: [93] WOA13 month...…”

Section: Rq4: Which Type Of Data Instance and Datasets Are Most Commo...mentioning

confidence: 99%

Applications of Generative Adversarial Networks in Anomaly Detection: A Systematic Literature Review

Sabuhi¹,

Zhou²,

Bezemer³

et al. 2021

Preprint

View full text Add to dashboard Cite

Anomaly detection has become an indispensable tool for modern society, applied in a wide range of applications, from detecting fraudulent transactions to malignant brain tumours. Over time, many anomaly detection techniques have been introduced. However, in general, they all suffer from the same problem: a lack of data that represents anomalous behaviour. As anomalous behaviour is usually costly (or dangerous) for a system, it is difficult to gather enough data that represents such behaviour. This, in turn, makes it difficult to develop and evaluate anomaly detection techniques. Recently, generative adversarial networks (GANs) have attracted a great deal of attention in anomaly detection research, due to their unique ability to generate new data. In this paper, we present a systematic literature review of the applications of GANs in anomaly detection, covering 128 papers on the subject. The goal of this review paper is to analyze and summarize: (1) which anomaly detection techniques can benefit from certain types of GANs, and how, (2) in which application domains GAN-assisted anomaly detection techniques have been applied, and (3) which datasets and performance metrics have been used to evaluate these techniques. Our study helps researchers and practitioners to find the most suitable GAN-assisted anomaly detection technique for their application. In addition, we present a research roadmap for future studies in this area.

show abstract

Robust and Computationally-Efficient Anomaly Detection Using Powers-Of-Two Networks

Cited by 11 publications

References 32 publications

Block Walsh–Hadamard Transform-based Binary Layers in Deep Neural Networks

Block Walsh–Hadamard Transform-based Binary Layers in Deep Neural Networks

Block Walsh-Hadamard Transform Based Binary Layers in Deep Neural Networks

Applications of Generative Adversarial Networks in Anomaly Detection: A Systematic Literature Review

Contact Info

Product

Resources

About