Towards a Framework for Measuring the Performance of a Security Operations Center Analyst

Abstract

The past few years have seen several studies reporting on the role of a Security Operations Center (SOC) analyst and metrics for assessing the performance of analysts. However, research suggests that analysts are dissatisfied with existing metrics as they fail to take into consideration several aspects of their tasks. Existing works advocate for research into this area. A major challenge to devising adequate metrics is that the real work of analysts that needs to be taken into consideration to assess their holistic performance has not been fully discussed. Furthermore, at present, there is no agreement on what constitutes core analysts' functions. Analysts' overall performance in a SOC could be obtained if there is a common agreement on the core functions upon which their performance can be evaluated. In this paper, we propose a framework depicting the core functions of analysts and KPIs that can be used to measure the performance of analysts. To do this, we conducted a thorough analysis of the functions of a SOC described in multiple sources of literature and engaged with several analysts and SOC managers from different industries using qualitative semi-structured interviews. Our research results identify the following: quality of analysts' analysis, quality of analysts' report, time-based measures and the absolute numbers derived from an analyst's tasks as the key performance indicators (KPIs) for assessing analysts' performance. We hope that our findings will stimulate more interest among cybersecurity researchers on assessment methods for analysts.

Hermes Ransomware v2.1 Action Monitoring using Next Generation Security Operation Center (NGSOC) Complex Correlation Rules

“…The 2016 Global Information Security Survey notes that SOC is available in 56% of the organizations surveyed [15]. The importance of unified, consolidated cybersecurity incident prevention, detection, and response [16] [17] will grow as more companies understand [18].…”

“…However, security organizations are increasingly aware of the importance and significance of integrating persons, methods, and technology as an integral component of SIEM. As a result, the SOC has grown to a new level of functionality, combining people, mechanisms, and technology [15], [16]. Now SOCs can manage longer and more complex initiatives, manage thousands of warnings and incidents daily, record and track violations, and transnational coordinate practices, resolving the issue of IT harmonization.…”

Development of a concept for cybersecurity metrics classification

“…The initial proposal and analysis of mathematical definitions of security indicators, such as «number of attacks», «minimum cost of attack», «maximum probability of attack» and even «attack surface» are given in [31]. The paper [32] proposes an initial framework for assessing system security by decomposing the system into security-sensitive components and assigning security ratings to each component. Summation of the scores of the components allows an assessment of system reliability.…”

Security Operations Center: A Systematic Study and Open Challenges

“…Standards & Guidelines [3], [30], [36], [60], [179] Security Audits & Maturity Assessments [2], [5], [63] Metrics [23], [30], [46], [57], [68], [81], [85], [163], [180]- [186] regulations. Additionally, the SOC team can help determine the IT risks for the company.…”

“…Historical performance metrics enable comparability between work-shifts or longer time periods [68]. Agyepong et al [85] conducted an extensive survey about performance metrics for SOCs and proposed a consecutive framework [186]. Examples: False positive rate [30], [68], average analysis time [68], readiness level [81], [181], Mean Time to Detect [185] • People metrics: To improve the performance of security analysts inside a SOC it is necessary to measure human activities and workflows [68].…”