Nitro: Hardware-Based System Call Tracing for Virtual Machines

Pfoh, Jonas; Schneider, Christian; Eckert, Claudia

doi:10.1007/978-3-642-25141-2_7

Cited by 71 publications

(45 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this way, the system call can be successfully re-executed when the handler returned. By reenabling system calls, we also guarantee a more transparent execution environment in contrast to earlier approaches [1], which disable system calls for the entire analysis process. This seems to be a subtle difference, but it can be exceptionally useful against attacks (e.g., [6], [7]) which generate invalid opcodes to detect virtualization or debuggers.…”

Section: B Extending New Blue Pill With Tracing Capabilitiesmentioning

confidence: 99%

“…As the #UD exception does not increases the instruction pointer (RIP), we do not have to bother with alignments to re-execute the SYSCALL. In contrast to [1], we enable here the EFER.SCE bit again for higher transparency. However, depending on the chosen transparencygranularity tradeoff, later we disable this bit again.…”

Section: Proposed System Call Tracing Methods For X64mentioning

confidence: 99%

“…Our new method, similarly to [1], is based on invalid opcode exceptions instead of page faults to mitigate the performance degradation problem of previous solutions. As a first step of our implementation, we unset the SCE (system call enable) bit of the guest's EFER register, which makes SYSCALL instructions unknown for the processor.…”

Section: Proposed System Call Tracing Methods For X64mentioning

confidence: 99%

“…While previous solutions offered these monitoring capabilities from the host [1], [2], or from another trusted VM [5], a dedicated environment (e.g., KVM, Xen) had to be installed for them. On the contrary, our approach enables monitoring without any interruption in the OS operation, which is suitable for systems that do not permit downtime.…”

Section: Related Workmentioning

confidence: 99%

“…We propose a new system monitoring approach that enhances either the transparency or performance of existing methods (e.g., Nitro [1] and Ether [2]) . In addition, our approach does not require to create a copy of the system to be analyzed, neither it requires the installation of analysis tools on the analyzed system itself.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Towards the automated detection of unknown malware on live systems

Pék

Buttyán

2014

2014 IEEE International Conference on Communications (ICC)

View full text Add to dashboard Cite

Abstract-In this paper, we propose a new system monitoring framework that can serve as an enabler for automated malware detection on live systems. Our approach takes advantage of the increased availability of hardware assisted virtualization capabilities of modern CPUs, and its basic novelty consists in launching a hypervisor layer on the live system without stopping and restarting it. This hypervisor runs at a higher privilege level than the OS itself, thus, it can be used to observe the behavior of the analyzed system in a transparent manner. For this purpose, we also propose a novel system call tracing method that is designed to be configurable in terms of transparency and granularity.

show abstract

Section: B Extending New Blue Pill With Tracing Capabilitiesmentioning

confidence: 99%

Section: Proposed System Call Tracing Methods For X64mentioning

confidence: 99%

Section: Proposed System Call Tracing Methods For X64mentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Towards the automated detection of unknown malware on live systems

Pék

Buttyán

2014

2014 IEEE International Conference on Communications (ICC)

View full text Add to dashboard Cite

show abstract

VAED: VMI‐assisted evasion detection approach for infrastructure as a service cloud

Mishra

Pilli

Varadharajan

et al. 2017

Concurrency and Computation

View full text Add to dashboard Cite

Cloud computing provides on demand provisioning of resources mostly offered as Infrastructure as a Service. The flexibility in services has opened doors for attackers. Research has been performed to detect various malware in the last few years. However, modern malware are advanced enough to detect the presence of virtualization environment, security analyzer, or even the hypervisor by observing the virtualization-specific information such as virtual processor features, timing features, etc. The malware exhibit evasive nature and can fool existing security solutions by performing modern antidetection tactics. In this paper, we propose an approach named as VMI-assisted evasion detection (VAED), deployed at virtual machine monitor, to detect the evasion-based malware attacks. The VAED is based on learning the program semantic of evasive malware. It uses system call dependency graph approach generated using Markov Chain principle and keeps track of system call ordering with transition probability distribution between each pair system calls. It uses software break point injection technique to extract the system call traces of evasive malware samples, which is free from any modification in hardware-specific values. Hence, it is secure from evasion attempts. The VAED is validated over evasive samples collected from the University of California on request, and results seem to be promising. KEYWORDS cloud security, intrusion detection, system call analysis, virtual machine introspection 1 advanced malware that can detect the presence of the security analyzer, virtualization environment, or hypervisor. 1 On detection of such analysis environment, malware change their behavior to thwart the detection. Moreover, some of these attacks hide their presence from tenant virtual machine (TVM) by manipulating the kernel-data structure. Hence, analysis environment incorrectly classifies such intrusions as benign applications. A compromised VM is a big threat to cloud infrastructure, which can breach security of other VM, virtual machine monitor (VMM), or Host OS, etc. Cloud service provider needs to ensure the security of their infrastructure and VMs of their customers. Signature-based analysis, static analysis, dynamic analysis, and VM introspection (VMI) approaches are popular intrusion detection techniques used by researchers in their security solutions. Signature analysis techniques maintain a database of known attack signatures. Static analysis techniques perform the reverse engineering on the binary code to understand the program-syntax behavior. Dynamic analysis techniques observe the Concurrency Computat: Pract Exper. 2017;29:e4133.wileyonlinelibrary.com/journal/cpe

show abstract

Detecting Kernel Control-Flow Modifying Rootkits

Wang¹,

Karri²

2013

Advances in Information Security

View full text Add to dashboard Cite

Nitro: Hardware-Based System Call Tracing for Virtual Machines

Cited by 71 publications

References 6 publications

Towards the automated detection of unknown malware on live systems

Towards the automated detection of unknown malware on live systems

VAED: VMI‐assisted evasion detection approach for infrastructure as a service cloud

Detecting Kernel Control-Flow Modifying Rootkits

Contact Info

Product

Resources

About