DeepReturn: A deep neural network can learn how to detect previously-unseen ROP payloads without using any heuristics

Liu, Xusheng; Hu, Zhisheng; Wang, Haizhou; Fu, Yiwei; Chen, Ping; Zhu, Minghui; Liu, Peng

doi:10.3233/jcs-191368

Cited by 4 publications

(15 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is also observed that preparing the data is the most challenging part for applying deep learning to detect ROP attacks. [Li et al, 2020, Zhang et al, 2019b. For example, proposed a unique data representation for traces acquired from Intel PT, which is a 2-dimensional grid data structure that can be used to training neural networks; Zhang et al [2019b] proposed a specialized fine-grained CFG and a unique way to create malicious data.…”

Section: Deep Learning Based Rop Detection Methodsmentioning

confidence: 99%

“…The flow diagram of the data preparation process is shown in Figure 2. The authors of DeepReturn, Li et al [2020] called the process of chaining up the instruction sequences Address Space Layout (ASL) guided disassembly. The detail of the ASL guided disassembly will be explained in Section 4.1.…”

Section: Domain Adaptationmentioning

confidence: 99%

“…To generate benign data for the training phase and extract instruction sequences from the incoming network data during the production phase, DeepReturn uses an approach called ASL-guided disassembly. This section summarizes the process of the ASL-guided disassembly proposed by Li et al [2020].…”

Section: Asl-guided Disassemblymentioning

confidence: 99%

“…An ROP detection task is essentially a classification task with two classes: benign or malicious. We observe that usually data with one of the two labels could be hard to prepare [Zhang et al, 2019b, Li et al, 2020. Therefore, if the time for generating the data is limited, one can choose to either have less total amount of data, or generate less amount of data with hard-to-prepare labels.…”

Section: Introductionmentioning

confidence: 98%

“…To mitigate the imbalanced data challenge in ROP detection, this paper introduces a transfer learning based solution to mitigate the imbalanced data issue for deep learning based ROP detection methods. The scenario in this paper is based on DeepReturn Li et al [2020], which has following assumptions:…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Tackling Imbalanced Data in Cybersecurity with Transfer Learning: A Case with ROP Payload Detection

Wang¹,

Liu²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

In recent years, deep learning gained proliferating popularity in the cybersecurity application domain, since when being compared to traditional machine learning, it usually involves less human effort, produces better results, and provides better generalizability. However, the imbalanced data issue is very common in cybersecurity, which can substantially deteriorate the performance of the deep learning models. This paper introduces a transfer learning based method to tackle the imbalanced data issue in cybersecurity using Return-Oriented Programming (ROP) payload detection as a case study. We achieved 0.033 average false positive rate, 0.9718 average F1 score and 0.9418 average detection rate on 3 different target domain programs using 2 different source domain programs, with 0 benign training data samples in the target domain. The performance improvement compared to the baseline is a trade-off between false positive rate and detection rate. Using our approach, the number of false positives is reduced by 23.20%, and as a trade-off, the number of detected malicious samples is reduced by 0.50%.

show abstract

Section: Deep Learning Based Rop Detection Methodsmentioning

confidence: 99%

Section: Domain Adaptationmentioning

confidence: 99%

Section: Asl-guided Disassemblymentioning

confidence: 99%

Section: Introductionmentioning

confidence: 98%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Tackling Imbalanced Data in Cybersecurity with Transfer Learning: A Case with ROP Payload Detection

Wang¹,

Liu²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

Tackling imbalanced data in cybersecurity with transfer learning: a case with ROP payload detection

2023

View full text Add to dashboard Cite

In recent years, deep learning gained proliferating popularity in the cybersecurity application domain, since when being compared to traditional machine learning methods, it usually involves less human efforts, produces better results, and provides better generalizability. However, the imbalanced data issue is very common in cybersecurity, which can substantially deteriorate the performance of the deep learning models. This paper introduces a transfer learning based method to tackle the imbalanced data issue in cybersecurity using return-oriented programming payload detection as a case study. We achieved 0.0290 average false positive rate, 0.9705 average F1 score and 0.9521 average detection rate on 3 different target domain programs using 2 different source domain programs, with 0 benign training data sample in the target domain. The performance improvement compared to the baseline is a trade-off between false positive rate and detection rate. Using our approach, the total number of false positives is reduced by 23.16%, and as a trade-off, the number of detected malicious samples decreases by 0.68%.

show abstract

Horus: An Effective and Reliable Framework for Code-Reuse Exploits Detection in Data Stream

Yang¹,

Liu²,

Tang³

2022

Preprint

View full text Add to dashboard Cite

Recent years have witnessed a rapid growth of code-reuse attacks in advance persistent threats and cyberspace crimes. Carefully crafted code-reuse exploits circumvent modern protection mechanisms and hijack the execution flow of a program to perform expected functionalities by chaining together existing codes. The sophistication and intrication of code-reuse exploits hinder the scrutinization and dissection on them. Although the previous literature has introduced some feasible approaches, effectiveness and reliability in practical applications remain severe challenges. To address this issue, we propose Horus, a data-driven framework for effective and reliable detection on code-reuse exploits. In order to raise the effectiveness against underlying noises, we comprehensively leverage the strengths of time-series and frequency-domain analysis, and propose a learning-based detector that synthesizes the contemporary twofold features. Then we employ a lightweight interpreter to speculatively and tentatively translate the suspicious bytes to open the black box and enhance the reliability and interpretability. Additionally, a functionality-preserving data augmentation is adopted to increase the diversity of limited training data and raise the generality for real-world deployment. Comparative experiments and ablation studies are conducted on a dataset composed of real-world instances to verify and prove the prevalence of Horus. The experimental results illustrate that Horus outperform existing methods on the identification of code-reuse exploits from data stream with an acceptable overhead. Horus does not rely on any dynamic executions and can be easily integrated into existing defense systems. Moreover, Horus is able to provide tentative interpretations about attack semantics irrespective of target program, which further improve system's effectiveness and reliability.

show abstract

DeepReturn: A deep neural network can learn how to detect previously-unseen ROP payloads without using any heuristics

Cited by 4 publications

References 34 publications

Tackling Imbalanced Data in Cybersecurity with Transfer Learning: A Case with ROP Payload Detection

Tackling Imbalanced Data in Cybersecurity with Transfer Learning: A Case with ROP Payload Detection

Tackling imbalanced data in cybersecurity with transfer learning: a case with ROP payload detection

Horus: An Effective and Reliable Framework for Code-Reuse Exploits Detection in Data Stream

Contact Info

Product

Resources

About