Elicitation of expert opinion is important for risk analysis when only limited data are available. Expert opinion is often elicited in the form of subjective confidence intervals; however, these are prone to substantial overconfidence. We investigated the influence of elicitation question format, in particular the number of steps in the elicitation procedure. In a 3-point elicitation procedure, an expert is asked for a lower limit, upper limit, and best guess, the two limits creating an interval of some assigned confidence level (e.g., 80%). In our 4-step interval elicitation procedure, experts were also asked for a realistic lower limit, upper limit, and best guess, but no confidence level was assigned; the fourth step was to rate their anticipated confidence in the interval produced. In our three studies, experts made interval predictions of rates of infectious diseases (Study 1, n = 21 and Study 2, n = 24: epidemiologists and public health experts), or marine invertebrate populations (Study 3, n = 34: ecologists and biologists). We combined the results from our studies using meta-analysis, which found average overconfidence of 11.9%, 95% CI [3.5, 20.3] (a hit rate of 68.1% for 80% intervals)-a substantial decrease in overconfidence compared with previous studies. Studies 2 and 3 suggest that the 4-step procedure is more likely to reduce overconfidence than the 3-point procedure (Cohen's d = 0.61, [0.04, 1.18]).

Expert judgements are essential when time and resources are stretched or we face novel dilemmas requiring fast solutions. Good advice can save lives and large sums of money. Typically, experts are defined by their qualifications, track record and experience [1], [2]. The social expectation hypothesis argues that more highly regarded and more experienced experts will give better advice. We asked experts to predict how they will perform, and how their peers will perform, on sets of questions. The results indicate that the way experts regard each other is consistent, but unfortunately, ranks are a poor guide to actual performance. Expert advice will be more accurate if technical decisions routinely use broadly-defined expert groups, structured question protocols and feedback.

People carrying germline mutations in mismatch repair genes are at high risk of colorectal cancer (CRC), yet about half of people from mutation-carrying families decline genetic counselling and/or testing to identify mutation status. We studied the association of quantitative measures of risk perception, risk prediction and self-reported screening colonoscopy in this elusive yet high-risk group. The sample of 26 participants (mean age 43.1 years, 14 women) in the Australasian Colorectal Cancer Family Registry were relatives of mutation carriers; had not been diagnosed with any cancer at the time of recruitment and had declined an invitation to attend genetic counselling and/or testing. A structured elicitation protocol captured perceived CRC risk over the next 10 years. Self-reported colonoscopy screening was elicited during a 45-minute semi-structured interview. Predicted 10-year CRC risk based on age, gender, known mutation status and family history was calculated using “MMRpro.” Mean perceived 10-year risk of CRC was 31% [95% CI 21, 40], compared with mean predicted risk of 4% [2, 7[ (p<0.001); this was independent of age and sex (p = 0.9). Among those reporting any medical advice and any screening colonoscopy (n = 18), those with higher risk perception had less frequent colonoscopy (Pearson’s r = 0.49 [0.02, 0.79]). People who decline genetic testing for CRC susceptibility mutations perceive themselves to be at substantially higher risk than they really are. Those with high perceived risk do not undertake screening colonoscopy more often than those who perceive themselves to be at average risk.

No abstract