Problems with Risk Matrices Using Ordinal Scales

Krisper, Michael

doi:10.48550/arxiv.2103.05440

Cited by 4 publications

(7 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To overcome this challenge, Krisper introduces different kinds of distributions, both numerically and graphically. Some common distributions of ranks are linear, logarithmic, normally distributed (Gaussian), and arbitrary (fitted) [66]. For instance, for each scenario in this paper, linear distributions of ranks were used.…”

Section: Discussionmentioning

confidence: 99%

Risk Analysis of Artificial Intelligence in Medicine with a Multilayer Concept of System Order

Moghadasi,

Valdez,

Piran

et al. 2024

Systems

View full text Add to dashboard Cite

Artificial intelligence (AI) is advancing across technology domains including healthcare, commerce, the economy, the environment, cybersecurity, transportation, etc. AI will transform healthcare systems, bringing profound changes to diagnosis, treatment, patient care, data, medicines, devices, etc. However, AI in healthcare introduces entirely new categories of risk for assessment, management, and communication. For this topic, the framing of conventional risk and decision analyses is ongoing. This paper introduces a method to quantify risk as the disruption of the order of AI initiatives in healthcare systems, aiming to find the scenarios that are most and least disruptive to system order. This novel approach addresses scenarios that bring about a re-ordering of initiatives in each of the following three characteristic layers: purpose, structure, and function. In each layer, the following model elements are identified: 1. Typical research and development initiatives in healthcare. 2. The ordering criteria of the initiatives. 3. Emergent conditions and scenarios that could influence the ordering of the AI initiatives. This approach is a manifold accounting of the scenarios that could contribute to the risk associated with AI in healthcare. Recognizing the context-specific nature of risks and highlighting the role of human in the loop, this study identifies scenario s.06—non-interpretable AI and lack of human–AI communications—as the most disruptive across all three layers of healthcare systems. This finding suggests that AI transparency solutions primarily target domain experts, a reasonable inclination given the significance of “high-stakes” AI systems, particularly in healthcare. Future work should connect this approach with decision analysis and quantifying the value of information. Future work will explore the disruptions of system order in additional layers of the healthcare system, including the environment, boundary, interconnections, workforce, facilities, supply chains, and others.

show abstract

Section: Discussionmentioning

confidence: 99%

Risk Analysis of Artificial Intelligence in Medicine with a Multilayer Concept of System Order

Moghadasi,

Valdez,

Piran

et al. 2024

Systems

View full text Add to dashboard Cite

show abstract

“…In the worst case, this can lead to a misguided investment in measures. To avoid this, an adjustment of the scoring to a quantitative metric is required (Krisper, 2021).…”

Section: Introductionmentioning

confidence: 99%

Risk Adjusting of Scoring-based Metrics in Physical Security Assessment

Termin,

Lichte,

Wolf

2023

Proceeding of the 33rd European Safety and Reliability Conference

View full text Add to dashboard Cite

Scoring-based systems are used worldwide to assess safety and security risks. Due to their ease of use, qualitative and semi-quantitative metrics are very popular. However, there may be the possibility that these scores do not accurately reflect the real risk, as was e.g. shown by Braband (2008) or Krisper (2021). In the worst case, this can lead to a misguided investment in measures. To avoid this, an adjustment of the scoring to a quantitative metric is required. The examples of the semi-quantitative Harnser metric and the quantitative vulnerability metric of Lichte et al. ( 2016) from physical security -in this work called intervention capability metric (ICM) -are used to show in this paper how to transfer a well-defined performance mechanism for quantitatively calculating physical vulnerability into consistent scores. To enable the transfer, this paper performs a metrical analysis. The results of the Harnser metric are extended by estimated probability intervals and compared to the results of the ICM. Different types of scales are used. Subsequently, we analyze measures to align the results of these two metrics, such as modifying the assignment of scores to scale categories or adjusting the probability intervals behind the scores. As an output, the metrical analysis generates rating scales for the Harnser scoring system that can be used to replicate quantitative vulnerability values. The results contribute to making more risk-appropriate decisions. Finally, we critically evaluate possibilities and limitations of metrical adaptability and summarize results.

show abstract

“…A certain interplay of the three performance mechanisms is required to achieve a protective effect. In the context of metric accuracy, Krisper (2021) and Termin et al (2021) analyze problems with the use of semi-quantitative approaches. In IT security, this performance mechanism is obviously missing.…”

Section: Introductionmentioning

confidence: 99%

An Analytic Approach to Analyze a Defense-in-Depth (DiD) Effect as proposed in IT Security Assessment

Termin¹,

Lichte²,

Wolf³

2022

Book of Extended Abstracts for the 32nd European Safety and Reliability Conference

View full text Add to dashboard Cite

Latest approaches in IT security assessments interpret the Common Vulnerability Scoring System (CVSS) parameters as barriers connected in series. In contrast to the classic multiplicative approach according to CVSS for determining exploitability via numerical values associated with the CVSS parameters, an additive approach is proposed in Braband (2019). Logarithmized CVSS scores are introduced to overcome the computational limitations with ordinal values. The log score sum across all barriers is sorted on a scale corresponding to a likelihood of exploitability (LoE) category. CVSS world is not only decomposed and remodeled into a mathematically admissible algorithm, but also contains an inherent defense-in-depth (DiD) effect. With each barrier added, the LoE decreases. This architectural interpretation can neither be falsified nor confirmed with previous CVSS metrics. Unlike in the IT security domain, tools exist in physical security to compute DiD in an objectively consistent manner. In our paper, we apply these considerations to a physical security setup in order to replicate his systemic modification based on CVSS. In a detailed analysis, we examine the boundary conditions and measures that must be taken in quantitative physical security metrics to emulate the DiD effect in IT security.

show abstract

Problems with Risk Matrices Using Ordinal Scales

Cited by 4 publications

References 17 publications

Risk Analysis of Artificial Intelligence in Medicine with a Multilayer Concept of System Order

Risk Analysis of Artificial Intelligence in Medicine with a Multilayer Concept of System Order

Risk Adjusting of Scoring-based Metrics in Physical Security Assessment

An Analytic Approach to Analyze a Defense-in-Depth (DiD) Effect as proposed in IT Security Assessment

Contact Info

Product

Resources

About