Experimentally Evaluating Bias-Reducing Visual Analytics Techniques in Intelligence Analysis

Kretz, Donald R.

doi:10.1007/978-3-319-95831-6_9

Cited by 4 publications

(3 citation statements)

References 57 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Before a vulnerability becomes public, it must still be scored (for example for bounty programs such as Bugcrowd 3 ) and there is limited empirical evidence on how such scoring is influenced by the competences of the assessor. Our tests revealed that assessment variability could be high, a result commonly emerged in many studies on opinion formation in a pool of experts, which motivated the development of methods for opinion debiasing (Kretz 2018), calibrating (Camerer and Johnson 1991;Lichtenstein et al 1982), and pooling (Dietrich and List 2017).…”

Section: Introductionmentioning

confidence: 80%

Measuring the accuracy of software vulnerability assessments: experiments with students and professionals

et al. 2020

View full text Add to dashboard Cite

Assessing the risks of software vulnerabilities is a key process of software development and security management. This assessment requires to consider multiple factors (technical features, operational environment, involved assets, status of the vulnerability lifecycle, etc.) and may depend from the assessor's knowledge and skills. In this work, we tackle with an important part of this problem by measuring the accuracy of technical vulnerability assessments by assessors with different level and type of knowledge. We report an experiment to compare how accurately students with different technical education and security professionals are able to assess the severity of software vulnerabilities with the Common Vulnerability Scoring System (v3) industry methodology. Our results could be useful for increasing awareness about the intrinsic subtleties of vulnerability risk assessment and possibly better compliance with regulations. With respect to academic education, professional training and human resources selections our work suggests that measuring the effects of knowledge and expertise on the accuracy of software security assessments is feasible albeit not easy.

show abstract

Section: Introductionmentioning

confidence: 80%

Measuring the accuracy of software vulnerability assessments: experiments with students and professionals

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Researchers have argued that computer tools can help overcome cognitive biases in decision making involving humanitarian scenarios [20,21]. However, we still lack comprehensive guidelines to avoid cognitive biases in decision making [45,68]. To the best of our knowledge, there is no existing review of user interfaces that funding managers use in such scenarios, and research on developing interfaces to support unbiased resource allocations is scarce.…”

Section: Design For Resource Allocationsmentioning

confidence: 99%

“…We identify that in the context of humanitarian resource allocation, prevalent cognitive biases stem from reasoning about people in need of aid as groups or communities, as opposed to considering each individual separately ( §2.1). However, how to curb these biases remains an open question [45].…”

Section: Introductionmentioning

confidence: 99%

Designing Resource Allocation Tools to Promote Fair Allocation: Do Visualization and Information Framing Matter?

Verma

Morais

Dragicevic

et al. 2023

Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

Studies on human decision-making focused on humanitarian aid have found that cognitive biases can hinder the fair allocation of resources. However, few HCI and Information Visualization studies have explored ways to overcome those cognitive biases. This work investigates whether the design of interactive resource allocation tools can help to promote allocation fairness. We specifically study the effect of presentation format (using text or visualization) and a specific framing strategy (showing resources allocated to groups or individuals). In our three crowdsourced experiments, we provided different tool designs to split money between two fictional programs that benefit two distinct communities. Our main finding indicates that individual-framed visualizations and text may be able to curb unfair allocations caused by group-framed designs. This work opens new perspectives that can motivate research on how interactive tools and visualizations can be engineered to combat cognitive biases that lead to inequitable decisions. CCS CONCEPTS• Human-centered computing → Empirical studies in visualization.

show abstract

Confirmation Bias in Analysts’ Response to Consensus Forecasts

Cai

Yao

Zhang

2022

Journal of Behavioral Finance

View full text Add to dashboard Cite

Experimentally Evaluating Bias-Reducing Visual Analytics Techniques in Intelligence Analysis

Cited by 4 publications

References 57 publications

Measuring the accuracy of software vulnerability assessments: experiments with students and professionals

Measuring the accuracy of software vulnerability assessments: experiments with students and professionals

Designing Resource Allocation Tools to Promote Fair Allocation: Do Visualization and Information Framing Matter?

Confirmation Bias in Analysts’ Response to Consensus Forecasts

Contact Info

Product

Resources

About