LogSed: Anomaly Diagnosis through Mining Time-Weighted Control Flow Graph in Logs

Jia, Tong; Yang, Lin; Chen, Pengfei; Liu, Ying; Meng, Fan Jing; Xu, Jie

doi:10.1109/cloud.2017.64

Cited by 55 publications

(21 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The authors in [7,8] introduced a classic log mining method to diagnose and locate anomalies in traditional distributed systems. According to the research in [9][10][11], the log mining method also plays an important role in the RCA of cloud applications. However, not all abnormal behaviors are recorded in logs: in many cases of RCA, in addition to log mining, O&M personnel have to combine their domain experience to find the root cause.…”

Section: Related Workmentioning

confidence: 99%

A Causality Mining and Knowledge Graph Based Method of Root Cause Diagnosis for Performance Anomaly in Cloud Applications

Qiu

Yin

et al. 2020

Applied Sciences

View full text Add to dashboard Cite

With the development of cloud computing technology, the microservice architecture (MSA) has become a prevailing application architecture in cloud-native applications. Many user-oriented services are supported by many microservices, and the dependencies between services are more complicated than those of a traditional monolithic architecture application. In such a situation, if an anomalous change happens in the performance metric of a microservice, it will cause other related services to be downgraded or even to fail, which would probably cause large losses to dependent businesses. Therefore, in the operation and maintenance job of cloud applications, it is critical to mine the causality of the problem and find its root cause as soon as possible. In this paper, we propose an approach for mining causality and diagnosing the root cause that uses knowledge graph technology and a causal search algorithm. We verified the proposed method on a classic cloud-native application and found that the method is effective. After applying our method on most of the services of a cloud-native application, both precision and recall were over 80%.

show abstract

Section: Related Workmentioning

confidence: 99%

A Causality Mining and Knowledge Graph Based Method of Root Cause Diagnosis for Performance Anomaly in Cloud Applications

Qiu

Yin

et al. 2020

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…For this reason, the solution has limited usability regarding full automation. Authors of [31] propose an approach to mine time-weighted graphs from logs with many threads running. The solution evaluated on the cloud environment performs with high f1-score that is about 80%.…”

Section: Mining Logs For Root Cause Classification and Diagnosticsmentioning

confidence: 99%

Next Stop "NoOps": Enabling Cross-System Diagnostics Through Graph-Based Composition of Logs and Metrics

Zasadzinski¹,

Solé²,

Brandon

et al. 2018

2018 IEEE International Conference on Cluster Computing (CLUSTER)

View full text Add to dashboard Cite

Performing diagnostics in IT systems is an increasingly complicated task, and it is not doable in satisfactory time by even the most skillful operators. Systems and their architecture change very rapidly in response to business and user demand. Many organizations see value in the maintenance and management model of NoOps that stands for No Operations. One of the implementations of this model is a system that is maintained automatically without any human intervention. The path to NoOps involves not only precise and fast diagnostics but also reusing as much knowledge as possible after the system is reconfigured or changed. The biggest challenge is to leverage knowledge on one IT system and reuse this knowledge for diagnostics of another, different system. We propose a framework of weighted graphs which can transfer knowledge, and perform high-quality diagnostics of IT systems. We encode all possible data in a graph representation of a system state and automatically calculate weights of these graphs. Then, thanks to the evaluation of similarity between graphs, we transfer knowledge about failures from one system to another and use it for diagnostics. We successfully evaluate the proposed approach on Spark, Hadoop, Kafka and Cassandra systems.

show abstract

“…Despite achieving promising results in terms of the detection precision and false positives, advanced attacks may penetrate the network to deliberately inject malicious code or even to disrupt the normal execution [4]. In addition, the legitimate software components-defined by the car manufactures during the design time-may exhibit abnormal behavior due to software/hardware malfunctioning.…”

Section: Introductionmentioning

confidence: 99%

“…Accordingly, the concept of solely observing the communication network is typically not sufficient for ensuring high-level of safety and security. Such a conclusion opened the door for developing several hostbased anomaly detection algorithms on the operating system level [5], [4], [6]. In this context, observing the computation time of the software components has been proposed to detect unexpected or suspicious behavior of such components [6].…”

Section: Introductionmentioning

confidence: 99%

AutoSec: Multidimensional Timing-Based Anomaly Detection for Automotive Cybersecurity

Tepic

Abdelaal

Weber

et al. 2020

2020 IEEE 26th International Conference on Embedded and Real-Time Computing Systems and Applications (RTCSA)

View full text Add to dashboard Cite

Nowadays, autonomous driving and driver assistance applications are being developed at an accelerated pace. This rapid growth is primarily driven by the potential of such smart applications to significantly improve safety on public roads and offer new possibilities for modern transportation concepts. Such indispensable applications typically require wireless connectivity between the vehicles and their surroundings, i.e. roadside infrastructure and cloud services. Nevertheless, such connectivity to external networks exposes the internal systems of individual vehicles to threats from remotely-launched attacks. In this realm, it is highly crucial to identify any misbehavior of the software components which might occur owing to either these threats or even software/hardware malfunctioning.In this paper, we introduce AutoSec, a host-based anomaly detection algorithm which relies on observing four timing parameters of the executed software components to accurately detect malicious behavior on the operating system level. To this end, AutoSec formulates the task of detecting anomalistic executions as a clustering problem. Specifically, AutoSec devises a hybrid clustering algorithm for grouping a set of collected timing traces resulted from executing the legitimate code. During the runtime, AutoSec simply classifies a certain execution as an anomaly, if its timing parameters are distant enough from the boundaries of the predefined clusters. To show the effectiveness of AutoSec, we collected timing traces from a testbed composed of a set of real and virtual control units communicating over a CAN bus. We show that using our proposed AutoSec, compared to baseline methods, we can identify up to 21% less false positives and 18% less false negatives.

show abstract

LogSed: Anomaly Diagnosis through Mining Time-Weighted Control Flow Graph in Logs

Cited by 55 publications

References 16 publications

A Causality Mining and Knowledge Graph Based Method of Root Cause Diagnosis for Performance Anomaly in Cloud Applications

A Causality Mining and Knowledge Graph Based Method of Root Cause Diagnosis for Performance Anomaly in Cloud Applications

Next Stop "NoOps": Enabling Cross-System Diagnostics Through Graph-Based Composition of Logs and Metrics

AutoSec: Multidimensional Timing-Based Anomaly Detection for Automotive Cybersecurity

Contact Info

Product

Resources

About