Categorizing Malware via A Word2Vec-based Temporal Convolutional Network Scheme

Abstract

As edge computing paradigm achieves great popularity in recent years, there remain some technical challenges that must be addressed to guarantee smart device security in Internet of Things (IoT) environment. Generally, smart devices transmit individual data across the IoT for various purposes nowadays, and it will cause losses and impose a huge threat to users since malware may steal and damage these data. To improve malware detection performance on IoT smart devices, we conduct a malware categorization analysis based on the Kaggle competition of Microsoft Malware Classification Challenge (BIG 2015) dataset in this article. Practically speaking, motivated by temporal convolutional network (TCN) structure, we propose a malware categorization scheme mainly using Word2Vec pre-trained model. Considering that the popular one-hot encoding converts input names from malicious files to high-dimensional vectors since each name is represented as one dimension in one-hot vector space, more compact vectors with fewer dimensions are obtained through the use of Word2Vec pre-training strategy, and then it can lead to fewer parameters and stronger malware feature representation. Moreover, compared with long short-term memory (LSTM), TCN demonstrates better performance with longer effective memory and faster training speed in sequence modeling tasks. The experimental comparisons on this malware dataset reveal better categorization performance with less memory usage and training time. Especially, through the performance comparison between our scheme and the state-of-the-art Word2Vec-based LSTM approach, our scheme shows approximately 1.3% higher predicted accuracy than the latter on this malware categorization task. Additionally, it also demonstrates that our scheme reduces about 90 thousand parameters and more than 1 hour on the model training time in this comparison.

NLP Technique for Malware Detection Using 1D CNN Fusion Model

“…A proposed automated malware detection framework MalDozer by Karbab et al [21] for example represented Android API method sequences in semantic vector forms using word2vec embedding and further employed a 2-dimensional convolutional neural network to extract features to train a model that was able to detect and classify malware at an F1-score of 96%-99%. Similarly, Sun et al [22] have shown in their proposed malware categorization framework how semantic vectors from word2vec can be combined with temporal convolutional network (TCN) for Windows malware detection. eir proposed word2vec-based TCN framework outperformed one-hot encoding-based TCN.…”

Transferable Features from 1D-Convolutional Network for Industrial Malware Classification

“…Hasegawa et al [18] used a 1D-convolutional network to detect Android malware. Then, to prevent information leakage from the future into the past, a specific temporal convolutional network (TCN) [19] was utilized to categorize malware into different families in the field of IoT malware classification [20].…”

Pruning and Quantization for Deep Neural Network Acceleration: A Survey

“…Convolutional neural networks are a class of feed-forward DNNs that use convolution operations to extract features from a data source. CNNs have been most successfully applied to visual-related tasks however they have found use in natural language processing [89], speech recognition [2], recommendation systems [204], malware detection [213] and industrial sensors time series prediction [251]. To provide a better understanding of optimization techniques, in this section, we introduce the two phases of CNN deployment -training and inference, discuss types of convolution operations, describe batch normalization (BN) as an acceleration technique for training, describe pooling as a technique to reduce complexity, and describe the exponential growth in parameters deployed in modern network structures.…”