Leveraging Reinforcement Learning and Generative Adversarial Networks to Craft Mutants of Windows Malware against Black-box Malware Detectors

Duy, Phan The; Luong, Tran Duc; An, Nguyen Hoang; Quyen, Nguyen Huu; Khoa, Nghi Hoang; Pham, Van-Hau

doi:10.1145/3568562.3568636

Cited by 5 publications

(2 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…B. Using AI for development-mutating malware that evades detection [15] Recent advancements in generative AI have led to the development of proof-of-concept models for polymorphic malware. These models demonstrate the potential of dynamically mutating malware payloads to evade detection and bypass endpoint and response (EDR) filters.…”

Section: Difficulties In Malware Detection and Incident Responsementioning

confidence: 99%

Enhancing Cyber-Resilience for Small and Medium-Sized Organizations with Prescriptive Malware Analysis, Detection and Response

Ilca,

Lucian,

Balan

2023

Sensors

View full text Add to dashboard Cite

In this study, the methodology of cyber-resilience in small and medium-sized organizations (SMEs) is investigated, and a comprehensive solution utilizing prescriptive malware analysis, detection and response using open-source solutions is proposed for detecting new emerging threats. By leveraging open-source solutions and software, a system specifically designed for SMEs with up to 250 employees is developed, focusing on the detection of new threats. Through extensive testing and validation, as well as efficient algorithms and techniques for anomaly detection, safety, and security, the effectiveness of the approach in enhancing SMEs’ cyber-defense capabilities and bolstering their overall cyber-resilience is demonstrated. The findings highlight the practicality and scalability of utilizing open-source resources to address the unique cybersecurity challenges faced by SMEs. The proposed system combines advanced malware analysis techniques with real-time threat intelligence feeds to identify and analyze malicious activities within SME networks. By employing machine-learning algorithms and behavior-based analysis, the system can effectively detect and classify sophisticated malware strains, including those previously unseen. To evaluate the system’s effectiveness, extensive testing and validation were conducted using real-world datasets and scenarios. The results demonstrate significant improvements in malware detection rates, with the system successfully identifying emerging threats that traditional security measures often miss. The proposed system represents a practical and scalable solution using containerized applications that can be readily deployed by SMEs seeking to enhance their cyber-defense capabilities.

show abstract

Section: Difficulties In Malware Detection and Incident Responsementioning

confidence: 99%

Enhancing Cyber-Resilience for Small and Medium-Sized Organizations with Prescriptive Malware Analysis, Detection and Response

Ilca,

Lucian,

Balan

2023

Sensors

View full text Add to dashboard Cite

show abstract

“…In the field of industrial IoT, Benaddi et al [43] focused on anomaly detection in Intrusion Detection Systems (IDS) using Distributional Reinforcement Learning and GAN. Phan et al [28] proposed an evasion method for black-box malware detectors to evaluate the effectiveness of RL and its combination with GAN, however the focus of their research work is only the comparison between both approaches and did not consider recent state-of-the-art evasion methods in their result analysis. Moreover, they targeted only blackdoor malware samples whereas we conducted our experiments on blackdoor and ransomware malware as ransomware attacks pose a serious threat in IoT devices [44].…”

Section: B Malware Evasion Techniquesmentioning

confidence: 99%

A Deep Reinforcement Learning Framework to Evade Black-Box Machine Learning Based IoT Malware Detectors Using GAN-Generated Influential Features

Arif,

Aslam,

Al-Otaibi

et al. 2023

IEEE Access

View full text Add to dashboard Cite

In the internet of things (IoT) networks, machine learning (ML) is significantly used for malware and adversary detection. Recently, research has shown that adversarial attacks have put ML-based models at risk. This problem is exacerbated in an IoT environment because of the absence of adequate security measures. Consequently, it is crucial to evaluate the strength of such malware detectors using powerful adversarial samples. The existing adversarial sample generation strategies either rely on high-level image features or an unfiltered feature set, making it challenging to determine which feature modifications are crucial in evading malware detection systems, without compromising the malware functionality. This encourages us to propose an evasion framework named IF-MalEvade, based on Generative Adversarial Network (GAN) and Deep Reinforcement Learning (DRL) that effectively generates fully-working, malware samples with several effective perturbations such as header section manipulation and benign bytes insertion. The DRL framework selects a few suitable action sequences to change malicious samples, thus allowing our malware samples to bypass various black-box ML based malware detectors and the detection search engines of VirusTotal, while maintaining the executability and malicious behavior of the original malware samples. The neural networks of GAN take in the unfiltered feature set of malware dataset and using minimax objective function yields a set of useful features that are subsequently used by the DRL agent to make effective changes. Experimental results illustrated that by utilizing the influential features in sequence of transformations, the adversarial samples generated by our model outperformed the state-of-the-art evasion models with an impressive evasion rate. Additionally, the detection rate of well-known machine learning models was also brought down to upto 97%. Furthermore, when the machine learning models were retrained using adversarial samples, a 35% increase in detection accuracy was observed.

show abstract

The Power of MEME: Adversarial Malware Creation with Model-Based Reinforcement Learning

Rigaki,

Garcia

2024

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Leveraging Reinforcement Learning and Generative Adversarial Networks to Craft Mutants of Windows Malware against Black-box Malware Detectors

Cited by 5 publications

References 11 publications

Enhancing Cyber-Resilience for Small and Medium-Sized Organizations with Prescriptive Malware Analysis, Detection and Response

Enhancing Cyber-Resilience for Small and Medium-Sized Organizations with Prescriptive Malware Analysis, Detection and Response

A Deep Reinforcement Learning Framework to Evade Black-Box Machine Learning Based IoT Malware Detectors Using GAN-Generated Influential Features

The Power of MEME: Adversarial Malware Creation with Model-Based Reinforcement Learning

Contact Info

Product

Resources

About