Anomaly Detection and Diagnosis Algorithms for Discrete Symbol Sequences with Applications to Airline Safety

Budalakoti, Suratna; Srivastava, Ashok N.; Otey, Matthew Eric

doi:10.1109/tsmcc.2008.2007248

Cited by 151 publications

(79 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…We are able to find a number of applications of anomaly detection in engineering problems, mainly fault tolerance [19][20][21][22][23]. An application in medical studies is work on blood oxygen saturation and heart rate in obstructive sleep apnea [24][25][26].…”

Section: Discussionmentioning

confidence: 99%

Generating Evidence Based Interpretation of Hematology Screens via Anomaly Characterization

David¹

2011

TOCCHEMJ

View full text Add to dashboard Cite

Introduction:We propose a simple, workable algorithm that provides assistance for interpreting any set of data from the screen of a blood analysis with high accuracy, reliability, and inter-operability with an electronic medical record. This has been made possible at least recently as a result of advances in mathematics, low computational costs, and rapid transmission of the necessary data for computation. Materials and Methods:The database used for this study is a file of 22,000 laboratory hemograms generated by two Beckman-Coulter Gen-S analyzers over a two month period in a 630 bed acute care facility in Brooklyn. All control samples, patient identifiers, and patients under 23 years old were stripped from the dataset. An experienced medical practitioner reviewed all of the data used in generating the algorithm described. The differential diagnoses were outlined prior to beginning the study, and preliminary studies were done to determine the reference ranges for each predictor. An algorithm for anomaly detection and classification via anomaly characterization is proposed. For each patient, the algorithm characterizes its anomalous profile and builds a differential metric to identify similar patients who are mapped into a classification. Results:The algorithm successfully classified patients into the diagnosis that were sufficient in sample size, and others are still under observation. The algorithm correctly classified the patients as follows: Microcytic Anemia -99.63%, Normocytic Anemia -98.03%, Mild SIRS -73.42%, Thrombocytopenia -99.52%, Leukocytopenia -84.83%, Moderate / Severe SIRS -96.69% and Normal -93.18%.Discussion: This limited analysis of automated hematological results can be extended to the case of more complicated conditions than presented, and can be extended to a combination of chemistry, hematology, immunology, and other data.

show abstract

Section: Discussionmentioning

confidence: 99%

Generating Evidence Based Interpretation of Hematology Screens via Anomaly Characterization

David¹

2011

TOCCHEMJ

View full text Add to dashboard Cite

show abstract

“…Other proposals that deal with the broader problem of outlier detection in data streams include detection of changes, e.g., [29]; consideration of discrete sequences, e.g., [30]; techniques that rely on estimating the deviation from the expected values in time-series, e.g., [31] and density, e.g., [32]; specialized techniques for sensor networks, e.g., [33], and probabilistic streams, e.g., [34,35]; and solutions for the high-dimensionality problem in streaming outlier detection, e.g., [36]. Distance-based outlier detection has been also considered in [37] without considering incremental outlier computation though, [38], which employs data editing techniques, and [39], which focuses on efficient correlation computation techniques for multiple time series.…”

Section: Symbolmentioning

confidence: 99%

Efficient and flexible algorithms for monitoring distance-based outliers over data streams

Kontaki

Gounaris

Papadopoulos

et al. 2016

Information Systems

View full text Add to dashboard Cite

“…One class of the methods is kernel-based methods. The most commonly used kernel method for discrete sequences is nLCS [18]. However, the kernel-based methods have to calculate the distance between all the training and test logs, thus they have high time complexity [6].…”

Section: Related Workmentioning

confidence: 99%

Online Anomaly Detection by Improved Grammar Compression of Log Sequences

Gao

Zhou²,

Zhang

et al. 2014

Proceedings of the 2014 SIAM International Conference on Data Mining

View full text Add to dashboard Cite

Nowadays, log sequences mining techniques are widely used in detecting anomalies for Internet services. The state-ofthe-art anomaly detection methods either need significant computational costs, or require specific assumptions that the test logs are holding certain data distribution patterns in order to be effective. Therefore, it is very difficult to achieve real time responses and it greatly reduces the effectiveness of these mechanisms in reality.To address these issues, we propose an innovative anomaly detection strategy called CADM. In CADM, the relative entropy between test logs and normal logs is exploited to discover the anomalous levels. Instead of calculating the relative entropy based on certain predefined data distribution models, our solution inspects the relationship between relative entropy and compression size with an improved grammar-based compression method. No assumptions are needed. In addition, our mechanism has excellent scalability with only O(n) computational complexity. It can generate the detection results on the fly. Experimental analysis with both synthetic and real world logs proves that CADM is superior to the other methods. It can achieve very high anomaly detection accuracy with the minimal computational overhead. It is suitable for log mining tasks and can be applied on a broad variety of application fields.

show abstract

Anomaly Detection and Diagnosis Algorithms for Discrete Symbol Sequences with Applications to Airline Safety

Cited by 151 publications

References 21 publications

Generating Evidence Based Interpretation of Hematology Screens via Anomaly Characterization

Generating Evidence Based Interpretation of Hematology Screens via Anomaly Characterization

Efficient and flexible algorithms for monitoring distance-based outliers over data streams

Online Anomaly Detection by Improved Grammar Compression of Log Sequences

Contact Info

Product

Resources

About