Reliable detection of episodes in event sequences

Gwadera, Robert; Atallah, Mikhail J.; Szpankowski, Wojciech

doi:10.1007/s10115-004-0174-5

Cited by 76 publications

(81 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Many researchers have studied on particular techniques, such as the statistical profiling using histograms [6], parametric statistical modeling [7], non-parametric statistical modeling [8], rulebased system [9], clustering-based technique [10], and spectral technique [11]. All these techniques are straightforward, but selecting appropriate parameters and threshold, especially when behavior of network traffic changes, is quite difficult.…”

mentioning

confidence: 99%

Performance of Interval-Based Features in Anomaly Detection by Using Machine Learning Approach

Limthong¹

2014

IJMLC

View full text Add to dashboard Cite

Abstract-Detecting various anomalies or unusual incidents in computer network traffic is one of the great challenges for both researchers and network administrators. If they had an efficient method that could detect network traffic anomalies quickly and accurately, they would be able to prevent security problems or network congestion caused by such anomalies. Therefore, we conducted a series of experiments to examine which and how interval-based network traffic features affect anomaly detection by using three famous machine learning algorithms: the naï ve Bayes, k-nearest neighbor, and support vector machine. Our findings would help researchers and network administrators to select effective interval-based features for each particular type of anomaly, and to choose a proper machine learning algorithm for their own network system. Index Terms-Network traffic, anomaly detection, naï veBayes, nearest neighbor, support vector machine. I. INTRODUCTIONOne of the crucial responsibilities of administrators is discovering various anomalies and unusual incidents in computer network system. Forms or causes of anomalies can vary considerably, which produce a variety of network problems such as network congestion or even security problems. Examples of network anomalies and unusual incidents are denial of service attacks (DoS), viruses or worms spreading, outages, misconfigurations, and flash crowds. If network administrators had an automatic mechanism that expeditiously detected unknown anomalies or unusual incidents, they would avoid serious consequences caused by such anomalies. Thus, an automatic mechanism detecting unknown anomalies in computer network traffic would be attractive.According to several studies [1]-[3], we can categorize detection methods into two major groups: signature-based methods and statistical-based methods.The signature-based methods, such as Snort [4], Suricata, or Bro [5], monitor and compare packets with predetermined attack patterns known as signatures. It is a simple and efficient method to examine network traffic. Although the false positive rate of this technique can also be low, comparing network packets or flows with a large set of signatures is a time consuming task and has limited predictive capabilities. In addition, the signature-based methods cannot detect novel anomalies that are not defined  Manuscript received June 15, 2013; revised December 25, 2013. This work was supported by the Faculty Members Development Scholarship Program of Bangkok University, Thailand.Kriangkrai Limthong is with the Department of Informatics, Graduate University of Advanced Studies (Sokendai), Japan (e-mail: kriangkrai.l@bu.ac.th).in signatures. It means that administrators have to update the system signatures frequently.The statistical-based methods, however, can learn behavior of network traffic and possibly detect novel anomalies and unusual incidents. Many researchers have studied on particular techniques, such as the statistical profiling using histograms [6], parametric statistical modeling [7], non-p...

show abstract

mentioning

confidence: 99%

Performance of Interval-Based Features in Anomaly Detection by Using Machine Learning Approach

Limthong¹

2014

IJMLC

View full text Add to dashboard Cite

show abstract

“…In our paper [8] the problem of the reliable detection of unusual episodes was investigated, where we considered an episode in the form of a single sequence occurring as an ordered subsequence of a large event stream within a window of a given fixed size. This kind of episode is called a "serial episode" in the terminology of [10], and we henceforth adopt this terminology.…”

Section: Introductionmentioning

confidence: 99%

“…This kind of episode is called a "serial episode" in the terminology of [10], and we henceforth adopt this terminology. In [8] we proposed a method for reliable detection of significant episodes, where as a measure of significance we used Ω ∃ (n, w, m) the number of windows of length w which contain at least one occurrence of serial episode S of length m as a subsequence in event sequence T after n shifts of the window. We proved that appropriately normalized Ω ∃ (n, w, m) has the Gaussian distribution, where the expected value E[Ω ∃ (n, w, m)] = nP ∃ (w, m) and P ∃ (w, m) is the probability that a serial episode S of length m occurs at least once in a window of length w in an event sequence T over an alphabet A.…”

Section: Introductionmentioning

confidence: 99%

“…The advantage of the Bernoulli model versus the first order Markov or higher is that for the Bernoulli model P ∃ (w, m) can be computed efficiently exploiting the structure of W ∃ (w, m) and the fact that the model requires only |A| probabilities of symbols of the alphabet A. Therefore in [8] we focused on the Bernoulli model for which we gave an efficient dynamic programming method for computing P ∃ (w, m). Using generating functions and complex asymptotics we presented an asymptotic approximation of P ∃ (w, m), which is of the form P ∃ (w, m) = 1 − Θ(ρ w ) for large w and 0 < ρ < 1.…”

Section: Introductionmentioning

confidence: 99%

“…We tested τ u (w) by artificially injecting "bad" episodes into the monitored event sequence and observed that τ u (w) did indeed provide a sharp detection of intentional (bad) episodes. Our paper [8] laid out foundations, but did not consider the case of detecting more than one serial episode simultaneously. This paper builds on [8] by extending it to the case of an arbitrary number of serial episodes, monitored simultaneously for an occurrence, including the important special case of all permutations of the same serial episode, called a "parallel episode" in the terminology of [10].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Detection of Significant Sets of Episodes in Event Sequences

Atallah

Szpankowski

Gwadera

Fourth IEEE International Conference on Data Mining (ICDM'04)

View full text Add to dashboard Cite

show abstract

A deep learning approach for predicting critical events using event logs

Huang

Deep

Zhou

et al. 2021

Quality & Reliability Eng

View full text Add to dashboard Cite

Event logs, comprising data on the occurrence of different types of events and associated times, are commonly collected during the operation of modern industrial machines and systems. It is widely believed that the rich information embedded in event logs can be used to predict the occurrence of critical events. In this paper, we propose a recurrent neural network model using time‐to‐event data from event logs not only to predict the time of the occurrence of a target event of interest, but also to interpret, from the trained model, significant events leading to the target event. To improve the performance of our model, sampling techniques and methods dealing with the censored data are utilized. The proposed model is tested on both simulated data and real‐world datasets. Through these comparison studies, we show that the deep learning approach can often achieve better prediction performance than the traditional statistical model, such as, the Cox proportional hazard model. The real‐world case study also shows that the model interpretation algorithm proposed in this work can reveal the underlying physical relationship among events.

show abstract

Reliable detection of episodes in event sequences

Cited by 76 publications

References 14 publications

Performance of Interval-Based Features in Anomaly Detection by Using Machine Learning Approach

Performance of Interval-Based Features in Anomaly Detection by Using Machine Learning Approach

Detection of Significant Sets of Episodes in Event Sequences

A deep learning approach for predicting critical events using event logs

Contact Info

Product

Resources

About