Evaluating Computer Intrusion Detection Systems

Milenkoski, Aleksandar; Vieira, Marco; Kounev, Samuel; Avritzer, Alberto; Payne, Bryan D.

doi:10.1145/2808691

Cited by 176 publications

(81 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…To the best of our knowledge, this is the first paper that aims at systematically understanding and analyzing the root causes of intrusion detection false-negatives, with the ultimate goal of drawing new insights and principles to guide the design of the next generation IDSs. This is true despite the recent resurgence in studying IDSs (see, for example, [2], [9], [11], [12], [13], [14] and the references therein). In a broad sense, the present study falls into the broader field of cybersecurity data analytics [15], [16], [17], [18], [19], [20].…”

Section: B Related Workmentioning

confidence: 99%

Analyzing Root Causes of Intrusion Detection False-Negatives: Methodology and Case Study

Ficke

Schweitzer

Bateman

et al. 2019

MILCOM 2019 - 2019 IEEE Military Communications Conference (MILCOM)

View full text Add to dashboard Cite

Intrusion Detection Systems (IDSs) are a necessary cyber defense mechanism. Unfortunately, their capability has fallen behind that of attackers. This motivates us to improve our understanding of the root causes of their false-negatives. In this paper we make a first step towards the ultimate goal of drawing useful insights and principles that can guide the design of next-generation IDSs. Specifically, we propose a methodology for analyzing the root causes of IDS false-negatives and conduct a case study based on Snort and a real-world dataset of cyber attacks. The case study allows us to draw useful insights.

show abstract

Section: B Related Workmentioning

confidence: 99%

Analyzing Root Causes of Intrusion Detection False-Negatives: Methodology and Case Study

Ficke

Schweitzer

Bateman

et al. 2019

MILCOM 2019 - 2019 IEEE Military Communications Conference (MILCOM)

View full text Add to dashboard Cite

show abstract

“…In order to examine the bias of the naïve estimators given by Eqs. (8)- (12), we need to investigate their asymptotic distributions. For this purpose, we make the following assumption:…”

Section: Investigating Statistical Properties Of Naïve Estimatorsmentioning

confidence: 99%

Statistical Estimation of Malware Detection Metrics in the Absence of Ground Truth

Sun

Chen

et al. 2018

IEEE Trans.Inform.Forensic Secur.

View full text Add to dashboard Cite

The accurate measurement of security metrics is a critical research problem, because an improper or inaccurate measurement process can ruin the usefulness of the metrics. This is a highly challenging problem, particularly when the ground truth is unknown or noisy. In this paper, we measure five malware detection metrics in the absence of ground truth, which is a realistic setting that imposes many technical challenges. The ultimate goal is to develop principled, automated methods for measuring these metrics at the maximum accuracy possible. The problem naturally calls for investigations into statistical estimators by casting the measurement problem as a statistical estimation problem. We propose statistical estimators for these five malware detection metrics. By investigating the statistical properties of these estimators, we characterize when the estimators are accurate, and what adjustments can be made to improve them under what circumstances. We use synthetic data with known ground truth to validate these statistical estimators. Then, we employ these estimators to measure five metrics with respect to a large data set collected from VirusTotal.

show abstract

“…Kührer et al [6] focused on the effectiveness of malware blacklists and showed that the current blacklist is insufficient to protect against the variety of malware threats. There are other studies focused on evaluating the strength of IDS or other security products [7,8], strength of user password [9,10], and so on. However, the effects of all the metrics mentioned above are not ideal when faced with unknown threats.…”

Section: Related Workmentioning

confidence: 99%

Security Measurement for Unknown Threats Based on Attack Preferences

Yin

Sun

Wang

et al. 2018

Security and Communication Networks

View full text Add to dashboard Cite

Security measurement matters to every stakeholder in network security. It provides security practitioners the exact security awareness. However, most of the works are not applicable to the unknown threat. What is more, existing efforts on security metric mainly focus on the ease of certain attack from a theoretical point of view, ignoring the "likelihood of exploitation." To help administrator have a better understanding, we analyze the behavior of attackers who exploit the zero-day vulnerabilities and predict their attack timing. Based on the prediction, we propose a method of security measurement. In detail, we compute the optimal attack timing from the perspective of attacker, using a long-term game to estimate the risk of being found and then choose the optimal timing based on the risk and profit. We design a learning strategy to model the information sharing mechanism among multiattackers and use spatial structure to model the long-term process. After calculating the Nash equilibrium for each subgame, we consider the likelihood of being attacked for each node as the security metric result. The experiment results show the efficiency of our approach.

show abstract

Evaluating Computer Intrusion Detection Systems

Cited by 176 publications

References 48 publications

Analyzing Root Causes of Intrusion Detection False-Negatives: Methodology and Case Study

Analyzing Root Causes of Intrusion Detection False-Negatives: Methodology and Case Study

Statistical Estimation of Malware Detection Metrics in the Absence of Ground Truth

Security Measurement for Unknown Threats Based on Attack Preferences

Contact Info

Product

Resources

About