Understanding Tradeoffs Between Throughput, Quality, and Cost of Alert Analysis in a CSOC

Shah, Ankit; Ganesan, Rajesh; Jajodia, Sushil; Çam, Hasan

doi:10.1109/tifs.2018.2871744

Cited by 18 publications

(17 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Given that it is the analyst that makes most of the final decisions during operations [6], it comes as no surprise that their performance is of interest to stakeholders and SOC managers [14]. In fact, Shah et al [20] explain that effective performance, such as the timely analysis of alert by the analysts is an essential characteristic of an efficient SOC. SOC managers and stakeholders, therefore, maintain a range of metrics and measures for the analysts.…”

Section: A the Role Of The Analystmentioning

confidence: 99%

Towards a Framework for Measuring the Performance of a Security Operations Center Analyst

Agyepong

Cherdantseva

Reinecke

et al. 2020

2020 International Conference on Cyber Security and Protection of Digital Services (Cyber Security)

View full text Add to dashboard Cite

The past few years have seen several studies reporting on the role of a Security Operations Center (SOC) analyst and metrics for assessing the performance of analysts. However, research suggests that analysts are dissatisfied with existing metrics as they fail to take into consideration several aspects of their tasks. Existing works advocate for research into this area. A major challenge to devising adequate metrics is that the real work of analysts that needs to be taken into consideration to assess their holistic performance has not been fully discussed. Furthermore, at present, there is no agreement on what constitutes core analysts' functions. Analysts' overall performance in a SOC could be obtained if there is a common agreement on the core functions upon which their performance can be evaluated. In this paper, we propose a framework depicting the core functions of analysts and KPIs that can be used to measure the performance of analysts. To do this, we conducted a thorough analysis of the functions of a SOC described in multiple sources of literature and engaged with several analysts and SOC managers from different industries using qualitative semi-structured interviews. Our research results identify the following: quality of analysts' analysis, quality of analysts' report, time-based measures and the absolute numbers derived from an analyst's tasks as the key performance indicators (KPIs) for assessing analysts' performance. We hope that our findings will stimulate more interest among cybersecurity researchers on assessment methods for analysts.

show abstract

Section: A the Role Of The Analystmentioning

confidence: 99%

Towards a Framework for Measuring the Performance of a Security Operations Center Analyst

Agyepong

Cherdantseva

Reinecke

et al. 2020

2020 International Conference on Cyber Security and Protection of Digital Services (Cyber Security)

View full text Add to dashboard Cite

show abstract

“…However, measuring how SOC operations respond to changes in the SOC configuration (e.g., a refined use-case, a different alert investigation process, new rulesets, etc.) remains an open and critical challenge [16], in terms of analyst competences as well as for metric definition and measurement [11,29,30,38].…”

Section: Security Monitoring Operationsmentioning

confidence: 99%

“…Research gap. Whereas current research has focused mostly on the identification of metrics [30] and procedures [15,29,30] to evaluate SOC performance, an empirical method capable of capturing the complexity of a SOC operation (including alert configuration, analyst capabilities, etc.) has yet to be proposed and validated [16].…”

Section: Soc Performance Evaluationmentioning

confidence: 99%

“…Whereas numerous performance metrics exist (e.g., number of detected incidents, time to detection, time to response, etc.) [30], it is still unclear how to obtain reliable and reproducible measures over those metrics. As incoming attacks are, by definition, unknown, a ground truth on which to base the measurements cannot be easily defined; on the other hand, performing measurements in in-vitro settings limits the representativeness of those measurements in real-world settings [16,29].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

SAIBERSOC: Synthetic Attack Injection to Benchmark and Evaluate the Performance of Security Operation Centers

Rosso

Campobasso

Gankhuyag

et al. 2020

Annual Computer Security Applications Conference

View full text Add to dashboard Cite

In this paper we introduce SAIBERSOC, a tool and methodology enabling security researchers and operators to evaluate the performance of deployed and operational Security Operation Centers (SOCs) (or any other security monitoring infrastructure). The methodology relies on the MITRE ATT&CK Framework to define a procedure to generate and automatically inject synthetic attacks in an operational SOC to evaluate any output metric of interest (e.g., detection accuracy, time-to-investigation, etc.). To evaluate the effectiveness of the proposed methodology, we devise an experiment with 𝑛 = 124 students playing the role of SOC analysts. The experiment relies on a real SOC infrastructure and assigns students to either a BADSOC or a GOODSOC experimental condition. Our results show that the proposed methodology is effective in identifying variations in SOC performance caused by (minimal) changes in SOC configuration. We release the SAIBERSOC tool implementation as free and open source software. CCS CONCEPTS• Security and privacy → Intrusion/anomaly detection and malware mitigation; Systems security.

show abstract

“…Over the years, the emerging technologies such as the social networks, Internet of Things (IoT), the fifth generation of communication (5G), the decentralized blockchain technologies, etc. have become an indispensable part of modern life [1], [2], [3], [4]. New technologies make our lives easier, faster, and more fun by creating amazing tools, devices, resources, and putting the most useful information at fingertips.…”

Section: Introductionmentioning

confidence: 99%

Blockchain-Based Digital Forensics Investigation Framework in the Internet of Things and Social Systems

Qin

Min

2019

IEEE Trans. Comput. Soc. Syst.

View full text Add to dashboard Cite

The decentralised nature of blockchain technologies can well match the needs of integrity and provenances of evidences collecting in digital forensics across jurisdictional borders. In this work, a novel blockchain based digital forensics investigation framework in the Internet of Things (IoT) and social systems environment is proposed, which can provide proof of existence and privacy preservation for evidence items examination. To implement such features, we present a block enabled forensics framework for IoT, namely IoT forensic chain (IoTFC), which can offer forensic investigation with good authenticity, immutability, traceability, resilience, and distributed trust between evidential entitles as well as examiners. The IoTFC can deliver a gurantee of traceability and track provenance of evidence items. Details of evidence identification, preservation, analysis, and presentation will be recorded in chains of block. The IoTFC can increase trust of both evidence items and examiners by providing transparency of the audit train. The use case demonstrated the effectiveness of proposed method.

show abstract

Understanding Tradeoffs Between Throughput, Quality, and Cost of Alert Analysis in a CSOC

Cited by 18 publications

References 22 publications

Towards a Framework for Measuring the Performance of a Security Operations Center Analyst

Towards a Framework for Measuring the Performance of a Security Operations Center Analyst

SAIBERSOC: Synthetic Attack Injection to Benchmark and Evaluate the Performance of Security Operation Centers

Blockchain-Based Digital Forensics Investigation Framework in the Internet of Things and Social Systems

Contact Info

Product

Resources

About