The Selection of Experts for (Probabilistic) Expert Knowledge Elicitation

Bolger, Fergus

doi:10.1007/978-3-319-65052-4_16

Cited by 20 publications

(20 citation statements)

References 77 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, group settings might induce experts to suppress dissent and the consideration of alternative solutions due to phenomenons such as group-think (Janis, 1972) or social conformity (Asch, 1956). The group elicitation result might also be more influenced by individuals with certain personality characteristics or a higher attributed seniority (Bolger, 2018;Gosling, 2018). On the other hand, if an experienced facilitator can counteract these biases, single experts may profit from a group discussion by acquiring relevant knowledge (Reagan-Cirincione, 1994).…”

Section: How Much Common Ground Should Be Established Before the Elicmentioning

confidence: 99%

“…The experts' predictions are then used to reconstruct the prior distribution that most closely matches these predictions, such as through the marginal likelihood formula in Equation 2. Importantly, indirect elicitation methods are designed to reduce the cognitive load on experts, thus making the elicitation procedure accessible for experts with less normative expertise (Bolger, 2018;Kadane, 1980). However, it is important to note that in addition to the many sources of method-ological flexibility discussed within this article, indirect approaches to prior specification contain several unique limitations.…”

Section: Which Prior Elicitation Technique?mentioning

confidence: 99%

See 1 more Smart Citation

Practical challenges and methodological flexibility in prior elicitation.

Stefan¹,

Evans²,

Wagenmakers³

2022

Psychological Methods

View full text Add to dashboard Cite

The Bayesian statistical framework requires the specification of prior distributions, which reflect predata knowledge about the relative plausibility of different parameter values. As prior distributions influence the results of Bayesian analyses, it is important to specify them with care. Prior elicitation has frequently been proposed as a principled method for deriving prior distributions based on expert knowledge. Although prior elicitation provides a theoretically satisfactory method of specifying prior distributions, there are several implicit decisions that researchers need to make at different stages of the elicitation process, each of them constituting important researcher degrees of freedom. Here, we discuss some of these decisions and group them into 3 categories: decisions about (a) the setup of the prior elicitation; (b) the core elicitation process; and (c) combination of elicited prior distributions from different experts. Importantly, different decision paths could result in greatly varying priors elicited from the same experts. Hence, researchers who wish to perform prior elicitation are advised to carefully consider each of the practical decisions before, during, and after the elicitation process. By explicitly outlining the consequences of these practical decisions, we hope to raise awareness for methodological flexibility in prior elicitation and provide researchers with a more structured approach to navigate the decision paths in prior elicitation. Making the decisions explicit also provides the foundation for further research that can identify evidence-based best practices that may eventually reduce the methodologically flexibility in prior elicitation.

show abstract

Section: How Much Common Ground Should Be Established Before the Elicmentioning

confidence: 99%

Section: Which Prior Elicitation Technique?mentioning

confidence: 99%

Practical challenges and methodological flexibility in prior elicitation.

Stefan¹,

Evans²,

Wagenmakers³

2022

Psychological Methods

View full text Add to dashboard Cite

show abstract

“…the values required to be quantified in the BN. Constructing a profile matrix can be useful (Bolger, 2018), which matches the knowledge requirements with the expert roles: it supports the identification of expertise needed as well as justification for the choice of experts. The number of required experts depends then on the variability of expertise per domain.…”

Section: The Four-stage Structured Elicitation Processmentioning

confidence: 99%

“…In our case, even if we had the availability of only one expert, we managed to satisfy an essential coverage of expertise in all relevant area. For larger projects, expert profile matrices can be useful at structuring this reflection (European Food Safety Authority, 2014) (Bolger, 2018).…”

Section: Evaluation Of the Processmentioning

confidence: 99%

An elicitation process to quantify Bayesian networks for dam failure analysis

et al. 2021

View full text Add to dashboard Cite

Bayesian networks support the probabilistic failure analysis of complex systems, e.g. dams and bridges, needed for a better understanding of the system reliability and for taking mitigation actions. Bayesian networks are useful in representing the interactions among system components graphically, while the quantitative strength of the interrelationships between the variables is measured using conditional probabilities. However, due to a lack of objective data it often becomes necessary to rely on expert judgment to provide subjective probabilities to quantify the model. This paper proposes an elicitation process that can be used to support the collection of valid and reliable data with the specific aim of quantifying a Bayesian Network, while minimizing the adverse impact of biases. To illustrate how this framework works, it is applied to a real-life case study regarding the safety of the Mountain Chute Dam and Generating Station, which is located on the Madawaska River in Ontario, Canada.

show abstract

“…This is not unexpected: as Lutz et al (2006) noted, despite factors that are likely to sustain the declining trend in the PTFR, several projection-makers anticipate instead a reversal of trends or some regression toward the mean. 8 These considerations emphasize the importance of the expert survey as a tool to broaden the information base and provide additional perspectives (Bolger 2018). Imagine in contrast what could result from a team of projection-makers in charge of developing assumptions for future fertility and who, after working in the same demographic projections unit for some time, tend to think along the same lines, either as the result of sharing the same influences or possibly due to some form of groupthink effect.…”

Section: Aggregation Of Individual Responsesmentioning

confidence: 99%

Using Expert Elicitation to Build Long-Term Projection Assumptions

Dion

Galbraith

Sirag

2020

Developments in Demographic Forecasting

View full text Add to dashboard Cite

Most statistical agencies consult with experts in some manner prior to formulating their assumptions about the future. Expert judgment is valuable when there is either a lack of good data, insufficient knowledge about underlying causal mechanisms, or apparent randomness in trends. In this paper, we describe the expert elicitation protocol developed by Statistics Canada in 2018 to inform the development of projection assumptions. The protocol may be useful for projection makers looking to adopt a formal approach to eliciting expert judgments, or for producing probabilistic projections, where it is necessary to obtain plausible estimates of uncertainty for components of population growth.

show abstract

The Selection of Experts for (Probabilistic) Expert Knowledge Elicitation

Cited by 20 publications

References 77 publications

Practical challenges and methodological flexibility in prior elicitation.

Practical challenges and methodological flexibility in prior elicitation.

An elicitation process to quantify Bayesian networks for dam failure analysis

Using Expert Elicitation to Build Long-Term Projection Assumptions

Contact Info

Product

Resources

About