Burstiness of Intrusion Detection Process: Empirical Evidence and a Modeling Approach

Abstract

Abstract-We analyze sets of intrusion detection records observed on the networks of several large, nonresidential organizations protected by a form of intrusion detection and prevention service. Our analyses reveal that the process of intrusion detection in these networks exhibits a significant degree of burstiness as well as strong memory, with burstiness and memory properties that are comparable to those of natural processes driven by threshold effects, but different from bursty human activities. We explore time-series models of these observable network security incidents based on partially observed data using a hidden Markov model with restricted hidden states, which we fit using Markov Chain Monte Carlo techniques. We examine the output of the fitted model with respect to its statistical properties and demonstrate that the model adequately accounts for intrinsic "bursting" within observed network incidents as a result of alternation between two or more stochastic processes. While our analysis does not lead directly to new detection capabilities, the practical implications of gaining better understanding of the observed burstiness are significant, and include opportunities for quantifying a network's risks and defensive efforts.Index Terms-bursty processes, cyber risk assessment, cyber security, estimating undetected malware, malware detection model, predictive models for cyber intrusions I. INTRODUCTION AND MOTIVATION A. MotivationWe explore approaches to modeling the detection process of cyber infections, i.e., presence of malicious softwaremalware -on a computer network. Often, this process is performed by a specialized organization -Managed Security Service Provider (MSSP) [1] -that monitors computer networks, analyzes the information obtained from the network, detects intrusions and activities of malware on the network, and reports such detections to the operators of the network who then take measures necessary to recover from the intrusion. As this research is based on empirical data provided to authors by a MSSP covering several large, non-residential networks, hereafter we use the term MSSP when referring to a network's cyber defenders.

Malware in the future? Forecasting of analyst detection of cyber events

“…Bursts in cyber attacks, however, are not a universal pattern. Using analyst verified reports, bursts of cyber attacks were found in only three out of five customer computers/networks protected by a CSSP [12]. Similarly, bursts were not reported for distributed denial-of-service (DDoS) attacks, but these data were limited to one-minute intervals over less than an hour [24].…”

“…Other potential applications for cyber event forecasting include proactive allocation of resources and capabilities for cyber defense (e.g., analyst staffing and sensor configuration) in CSSPs. Enhanced threat awareness may improve cybersecurity by helping to optimize human and technical capabilities for cyber defense.In contrast, nearly all prior research on modeling cyber attacks [4-10] lacked analyst detection and verification of computer security incidents with the exceptions of [11,12]. In these two exceptions, security incidents were verified by system administrators at a large university [11] or verified by analysts at a CSSP [12].…”

NoDoze: Combatting Threat Alert Fatigue with Automated Provenance Triage

“…Even if the TDS identifies an anomalous event related to the attack, cyber analysts are barraged with alerts on a daily basis and face the problem of finding a "needle in a haystack". Existing automated TDS are notorious for generating a high amount of false alarms [59], [2], [6], [34], [21]. Cyber analysts are in short supply, so organizations face a key challenge in managing the enormous volume of alerts they receive using the limited time of analysts [4].…”

“…Researchers have also proposed to automatically detect attacks using machine learning [38], [64]. However, these methods also have significant detection error and suffer from generating too many false alerts [58], [34].…”

A Markov-Based Intrusion Tolerance Finite Automaton

“…Specifically, the system was treated as a statistical Markov model containing a set of observable states and a set of hidden states, and the model was used to effectively detect the intrusion behavior in the network, highlighting the keys in the maintenance of the tolerance system. To prevent illegal intrusion into the network of large non-res-F. Li idential organizations, Harang and Kott [3] proposed a hidden Markov model with restricted hidden state, which couples Markov chain with Monte-Carlo simulation. By analyzing the combination of intrusion time series, the proposed model parses and predicts the network risks and provides the defense measures.…”