Integrating expert knowledge with data in Bayesian networks: Preserving data-driven expectations when the expert variables remain unobserved

Abstract

When developing a causal probabilistic model, i.e. a Bayesian network (BN), it is common to incorporate expert knowledge of factors that are important for decision analysis but where historical data are unavailable or difficult to obtain. This paper focuses on the problem whereby the distribution of some continuous variable in a BN is known from data, but where we wish to explicitly model the impact of some additional expert variable (for which there is expert judgment but no data). Because the statistical outcomes are already influenced by the causes an expert might identify as variables missing from the dataset, the incentive here is to add the expert factor to the model in such a way that the distribution of the data variable is preserved when the expert factor remains unobserved. We provide a method for eliciting expert judgment that ensures the expected values of a data variable are preserved under all the known conditions. We show that it is generally neither possible, nor realistic, to preserve the variance of the data variable, but we provide a method towards determining the accuracy of expertise in terms of the extent to which the variability of the revised empirical distribution is minimised. We also describe how to incorporate the assessment of extremely rare or previously unobserved events.

Learning from Both Experts and Data

“…This property therefore provides us, in these cases, with an explicit expression of a solution of (7) which was not otherwise trivial to obtain by a direct calculation looking for the saddle points of the Lagrangian (for example in the case i = j = 1). Proposition 4.3 Letp i,j n defined by (7) then for all i = j, it existsp ∈p i,j n such that ∃α n ∈ [0, 1]: p = α n p expert + (1 − α n )p emp n (9) where α n = n p emp n − p expert i if n ≤ p emp n − p expert i and α n = 1 otherwise.…”

Elemental patterns of verification strategies

“…Prior probabilities can be elicited from historical data and/or experts. A considerable amount of research has been done in eliciting beliefs from experts, whereas research addressing elicitation of beliefs from data in the context of BNs is limited …”

“…In cases where rich data are available, priors can be directly elicited from data. However, the elicitor must approach the data with caution as often historical data are prone to biases . In addition, care needs to be taken to avoid confounding the data sources for prior probabilities and for conditional probabilities of the event nodes.…”

Identification of I‐equivalent subnetworks in Bayesian networks to incorporate experts' knowledge

“…However, Constantinou, Fenton, and Neil () pointed out that many BN models have been constructed from either only data or only expert knowledge because of the difficulty of incorporating expert knowledge in learning BNs. Some studies have used expert knowledge for constructing an initial structure or providing a prior probability (Andreassen, Riekehr, Kristensen, Schønheyder, & Leibovici, ; Heckerman & Nathwani, ; Lucas, de Bruijn, Schurink, & Hoepelman, ; Werhli & Husmeier, ).…”