Identifying Useful Features for Malware Detection in the Ember Dataset

Oyama, Yoshihiro; Miyashita, Takumi; Kokubo, Hirotaka

doi:10.1109/candarw.2019.00069

Cited by 26 publications

(9 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As with Malconv, adding strings in the binary file is an effective technique to perturb Ember. This lack of robustness is consistent with the study of Oyama et al [35] which explains that only a few features contribute to explain Ember's results.…”

Section: Reinforce Algorithmsupporting

confidence: 91%

MERLIN -- Malware Evasion with Reinforcement LearnINg

Quertier¹,

Marais²,

Morucci³

et al. 2022

Preprint

View full text Add to dashboard Cite

In addition to signature-based and heuristics-based detection techniques, machine learning (ML) is widely used to generalize to new, never-before-seen malicious software (malware). However, it has been demonstrated that ML models can be fooled by tricking the classifier into returning the incorrect label. These studies, for instance, usually rely on a prediction score that is fragile to gradient-based attacks. In the context of a more realistic situation where an attacker has very little information about the outputs of a malware detection engine, modest evasion rates are achieved [1]. In this paper, we propose a method using reinforcement learning with DQN and REINFORCE algorithms to challenge two state-of-the-art ML-based detection engines (MalConv & EMBER) and a commercial antivirus (AV) classified by Gartner as a leader AV [2]. Our method combines several actions, modifying a Windows portable execution (PE) file without breaking its functionalities. Our method also identifies which actions perform better and compiles a detailed vulnerability report to help mitigate the evasion. We demonstrate that REINFORCE achieves very good evasion rates even on a commercial AV with limited available information.

show abstract

Section: Reinforce Algorithmsupporting

confidence: 91%

MERLIN -- Malware Evasion with Reinforcement LearnINg

Quertier¹,

Marais²,

Morucci³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…During this research for performance evaluation of the network following evaluation, metrics are considered. Researchers most commonly use these measures for assessing performance [53,54,55].…”

Section: Resultsmentioning

confidence: 99%

Improved Deep Learning Model for Static PE Files Malware Detection and Classification

Lad.¹,

Adamuthe²

2022

IJCNIS

View full text Add to dashboard Cite

Static analysis and detection of malware is a crucial phase for handling security threats. Most researchers stated that the problem with the static analysis is an imbalance in the dataset, causing invalid result metrics. It requires more time for extracting features from the raw binaries, and methods like neural networks require more time for the training. Considering these problems, we proposed a model capable of building a feature set from the dataset and classifying static PE files efficiently. The research work was conducted to emphasize the importance of feature extraction rather than focusing on model building. The well-extracted features help to provide better results when fed to neural networks with minimal numbers of layers. Using minimum layers will enhance the performance of the model and take fewer resources and time for the processing and evaluation. In this research work, EMBER datasets published by Endgame Inc. containing PE file information are used. Feature extraction, data standardization, and data cleaning techniques are performed to handle the imbalance and impurities from the dataset. Later the extracted features were scaled into a standard form to avoid the problems related to range variations. A total of 2381 features are extracted and pre-processed from both the 2017 and 2018 datasets, respectively. The pre-processed data is then given to a deep learning model for training. The deep learning model created using dense and dropout layers to minimize the resource strain on the model and deliver more accurate results in less amount of time. The results obtained during experimentation for EMBER v2017 and v2018 datasets are 97.53% and 94.09%, respectively. The model is trained for ten epochs with a learning rate of 0.01, and it took 4 minutes/epoch, which is one minute lesser than the Decision Tree model. In terms of precision metrics, our model achieved 98.85%, which is 1.85% more as compared to the existing models.

show abstract

“…Results obtained in the previous model release reported a 93% detection rate [24], meaning that the EMBER dataset developers have successfully hardened the process of correctly classifying malicious samples. Despite more recent works on the same dataset provide small improvements in the detection rate [25,26], they usually rely on deep learning frameworks that make more difficult to interpret model outputs.…”

Section: Related Workmentioning

confidence: 99%

Towards an Automated Pipeline for Detecting and Classifying Malware through Machine Learning

Loi¹,

Borile²,

Ucci³

2021

Preprint

View full text Add to dashboard Cite

The constant growth in the number of malware -software or code fragment potentially harmful for computers and information networks -and the use of sophisticated evasion and obfuscation techniques have seriously hindered classic signature-based approaches. On the other hand, malware detection systems based on machine learning techniques started offering a promising alternative to standard approaches, drastically reducing analysis time and turning out to be more robust against evasion and obfuscation techniques. In this paper, we propose a malware taxonomic classification pipeline able to classify Windows Portable Executable files (PEs). Given an input PE sample, it is first classified as either malicious or benign. If malicious, the pipeline further analyzes it in order to establish its threat type, family, and behavior(s). We tested the proposed pipeline on the open source dataset EMBER, containing approximately 1 million PE samples, analyzed through static analysis. Obtained malware detection results are comparable to other academic works in the current state of art and, in addition, we provide an in-depth classification of malicious samples. Models used in the pipeline provides interpretable results which can help security analysts in better understanding decisions taken by the automated pipeline.

show abstract

Identifying Useful Features for Malware Detection in the Ember Dataset

Cited by 26 publications

References 11 publications

MERLIN -- Malware Evasion with Reinforcement LearnINg

MERLIN -- Malware Evasion with Reinforcement LearnINg

Improved Deep Learning Model for Static PE Files Malware Detection and Classification

Towards an Automated Pipeline for Detecting and Classifying Malware through Machine Learning

Contact Info

Product

Resources

About