No abstract

The number of security incidents in computer systems is steadily increasing, despite intrusion detection and prevention mechanisms deployed as countermeasures. Many existing intrusion detection and prevention systems struggle to keep up with new threats posed by zero-day attacks and/or have serious performance impacts through extensive monitoring, questioning their effectiveness in most real-life scenarios. In this paper, we present a new approach for reactive security monitoring in a virtualized computer environment based on minimally-intrusive dynamic sensors deployed vertically across virtualization layers and horizontally within a virtual machine instance. The sensor streams are analyzed using a novel federation of complex event processing engines and an optimized query index to maximize the performance of continuous queries, and the results of the analysis are used to trigger appropriate actions on different virtualization layers in response to detected security anomalies. Furthermore, a novel event store that supports fast event logging is utilized for offline analysis of collected historical data. Experiments show that the proposed system can execute tens of thousands of complex, stateful detection rules simultaneously and trigger actions efficiently and with low latency.

Reactive security monitoring, self-driving cars, the Internet of Things (IoT), and many other novel applications require systems for both writing events arriving at very high and fluctuating rates to persistent storage as well as supporting analytical ad hoc queries. As standard database systems are not capable of delivering the required write performance, log-based systems, key-value stores, and other write-optimized data stores have emerged recently. However, the drawbacks of these systems are a fair query performance and the lack of suitable instant recovery mechanisms in case of system failures. In this article, we present ChronicleDB, a novel database system with a storage layout tailored for high write performance under fluctuating data rates and powerful indexing capabilities to support a variety of queries. In addition, ChronicleDB offers low-cost fault tolerance and instant recovery within milliseconds. Unlike previous work, ChronicleDB is designed either as a serverless library to be tightly integrated in an application or as a standalone database server. Our results of an experimental evaluation with real and synthetic data reveal that ChronicleDB clearly outperforms competing systems with respect to both write and query performance.

No abstract