Evaluation of the Diagnostic Capabilities of Commercial Intrusion Detection Systems

Debar, Hervé; Morin, Benjamin

doi:10.1007/3-540-36084-0_10

Cited by 22 publications

(18 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[3] reports about 10.000 false alarms per month for the use of commercial intrusion detection systems. Other evaluations [2], [5], confirm this experience. Small false positive rates are an important presumption for the acceptance of misuse detection systems in practice.…”

Section: On the Derivation Of Signaturessupporting

confidence: 73%

See 1 more Smart Citation

Towards Systematic Signature Testing

Schmerl

Koenig

2007

Testing of Software and Communicating Systems

View full text Add to dashboard Cite

Abstract. The success and the acceptance of intrusion detection systems essentially depend on the accuracy of their analysis. Inaccurate signatures strongly trigger false alarms. In practice several thousand false alarms per month are reported which limit the successful deployment of intrusion detection systems. Most today deployed intrusion detection systems apply misuse detection as detection procedure. Misuse detection compares the recorded audit data with predefined patterns, the signatures. These are mostly empirically developed based on experience and knowledge of experts. Methods for a systematic development have been scarcely reported yet. A testing and correcting phase is required to improve the quality of the signatures. Signature testing is still a rather empirical process like signature development itself. There exists no test methodology so far. In this paper we present first approaches for a systematic test of signatures. We characterize the test objectives and present different test methods. MotivationThe increasing dependence of human society on information technology (IT) systems requires appropriate measures to cope with their misuse. The enlarging technological complexity of IT systems increases the range of threats to endanger them. Besides preventive security measures reactive approaches are more and more applied to counter these threats. Reactive approaches allow responses and counter measures when security violations happened to prevent further damage. Complementary to preventive measures intrusion detection and prevention systems have proved as important means to protect IT resources. Meanwhile a wide range of commercial intrusion detection products is offered, especially for misuse detection. Nevertheless intrusion detection systems (IDSs) are not still deployed in a large scale. The reason is that the technology is considered not matured enough. Lacking reliability often resulting in high false alarm rates questions the practicability of intrusion detection systems [9].The security function intrusion detection deals with the monitoring of IT systems to detect security violations. The decision which activities have to be considered as security violations in a given context is defined by the applied security policy. Two main complementary approaches are applied: anomaly and misuse detection. Anomaly detection aims at the exposure of abnormal user behavior. It requires a comprehensive

show abstract

Section: On the Derivation Of Signaturessupporting

confidence: 73%

“…(1) to transfer the attacked system in a vulnerable state, (2) to exploit the vulnerability to intrude the system, and (3) to access to the compromised system and/or to change its system data. (This is the proper objective of the attack.…”

Section: On the Derivation Of Signaturesmentioning

confidence: 99%

Towards Systematic Signature Testing

Schmerl

Koenig

2007

Testing of Software and Communicating Systems

View full text Add to dashboard Cite

show abstract

“…By design, Bro reports all matching signatures, but each one only once per connection. This is similar to the approach suggested in [10]. Snort, on the other hand, reports the first matching signature for each packet, independently of the connection it belongs to.…”

Section: Performance Evaluationmentioning

confidence: 88%

“…[14] further extends the evaluation method by providing a user-friendly environment on the one hand, and new characterizations of attack traffic on the other hand. More recently, [10] evaluates several commercial systems, emphasizing the view of an analyst who receives the alerts, finding that these systems ignore relevant information about the context of the alerts. [15] discusses developing a benchmark for NIDSs, measuring their capacity with a representative traffic mix.…”

Section: Related Workmentioning

confidence: 99%

Enhancing byte-level network intrusion detection signatures with context

Sommer¹,

Paxson

2003

Proceedings of the 10th ACM Conference on Computer and Communications Security

191

View full text Add to dashboard Cite

Many network intrusion detection systems (NIDS) use byte sequences as signatures to detect malicious activity. While being highly efficient, they tend to suffer from a high false-positive rate. We develop the concept of contextual signatures as an improvement of string-based signature-matching. Rather than matching fixed strings in isolation, we augment the matching process with additional context. When designing an efficient signature engine for the NIDS Bro, we provide low-level context by using regular expressions for matching, and high-level context by taking advantage of the semantic information made available by Bro's protocol analysis and scripting language. Therewith, we greatly enhance the signature's expressiveness and hence the ability to reduce false positives. We present several examples such as matching requests with replies, using knowledge of the environment, defining dependencies between signatures to model step-wise attacks, and recognizing exploit scans.To leverage existing efforts, we convert the comprehensive signature set of the popular freeware NIDS Snort into Bro's language. While this does not provide us with improved signatures by itself, we reap an established base to build upon. Consequently, we evaluate our work by comparing to Snort, discussing in the process several general problems of comparing different NIDSs.

show abstract

“…Apart from testing data sets, some methodologies [4,43], frameworks [23,24] and tools [34,42] for assessing IDSs have also been proposed up to now. Additionally, comparative studies on commercial IDSs have been presented, mainly in network journals.…”

Section: Introductionmentioning

confidence: 99%

Mutating network scans for the assessment of supervised classifier ensembles

Sedano

González²,

Herrero³

et al. 2012

Logic Journal of IGPL

View full text Add to dashboard Cite

As it is well known, some Intrusion Detection Systems (IDSs) suffer from high rates of false positives and negatives. A mutation technique is proposed in this study to test and evaluate the performance of a full range of classifier ensembles for Network Intrusion Detection when trying to recognize new attacks. The novel technique applies mutant operators that randomly modify the features of the captured network packets to generate situations that could not otherwise be provided to IDSs while learning. A comprehensive comparison of supervised classifiers and their ensembles is performed to assess their generalization capability. It is based on the idea of confronting brand new network attacks obtained by means of the mutation technique. Finally, an example application of the proposed testing model is specially applied to the identification of network scans and related mutations.

show abstract

Evaluation of the Diagnostic Capabilities of Commercial Intrusion Detection Systems

Cited by 22 publications

References 6 publications

Towards Systematic Signature Testing

Towards Systematic Signature Testing

Enhancing byte-level network intrusion detection signatures with context

Mutating network scans for the assessment of supervised classifier ensembles

Contact Info

Product

Resources

About