Entirely possible overruns: how people think and talk about probabilistic cost estimates

Abstract

Purpose The purpose of this paper is to examine people’s understanding and evaluation of uncertainty intervals produced by experts as part of a quality assurance procedure of large public projects. Design/methodology/approach Three samples of educated participants (employees in a large construction company, students attending courses in project management and judgment and decision making, and judges of district and appeal courts) answered questionnaires about cost estimates of a highway construction project, presented as a probability distribution. Findings The studies demonstrated additivity neglect of probabilities that are graphically displayed. People’s evaluations of the accuracy of interval estimates revealed a boundary (a “cliff”) effect, with a sharp drop in accuracy ratings for outcomes above an arbitrary maximum. Several common verbal phrases (what “can” happen, is “entirely possible” and “not surprising”) which might seem to indicate expected outcomes were regularly used to describe unlikely values near or at the top of the distribution (an extremity effect). Research limitations/implications All judgments concerned a single case and were made by participants who were not stakeholders in this specific project. Further studies should compare judgments aided by a graph with conditions where the graph is changed or absent. Practical implications Experts and project managers cannot assume that readers of cost estimates understand a well-defined uncertainty interval as intended. They should also be aware of effects created by describing uncertain estimates in words. Originality/value The studies show how inconsistencies in judgment affect the understanding and evaluation of uncertainty intervals by well-informed and educated samples tested in a maximally transparent situation. Readers of cost estimates seem to believe that precise estimates are feasible and yet that costs are usually underestimated.

People prefer to predict middle, most likely quantitative outcomes (not extreme ones), but they still over-estimate their likelihood

“…The extremity preference seems to be a robust phenomenon replicated across different samples (e.g., British, American, Norwegian), and with different methods of distribution presentation (e.g., graph, table, text; Jenkins et al, 2018 ; Juanchich et al, 2013 ). The extremity preference has also been replicated over a variety of contexts, such as daily life events ( Juanchich et al, 2013 ; Teigen et al, 2013 , 2014 ), business project completion time ( Løhre & Teigen, 2014 ), construction cost estimates ( Teigen et al, 2019 ), and natural disasters ( Jenkins et al, 2018 ; Juanchich & Sirota, 2017 ; Teigen et al, 2018 ). However, the preference for extreme outcomes may not be independent of the verbal expression being used.…”

Dimensions of uncertainty communication: What is conveyed by verbal terms and numeric ranges

“…People also selected high values to indicate what has a chance to happen, is uncertain, is possible, and may or could be the case (Teigen et al, 2018a(Teigen et al, , 2018b). An entirely possible outcome is in this context not an outcome with a probability around 50%, as some translation studies indicate (e.g., Lichtenstein & Newman, 1967;Reagan et al, 1989), but rather the topmost outcome, which in a bell-shaped distribution has a very low probability of occurring (Teigen et al, 2019). When asked to describe an unlikely or improbable outcome, speakers typically selected an extreme value that had yet to be observed, located outside of the range of expected results.…”

Quantifying heterogeneous individual perceptions in project management research

“…Some of the these articles ask standard closed-ended survey questions to measure perceptions on nominal (e.g. Teigen et al, 2019), interval (e.g. Chaudhry et al, 2019;Keil et al, 2000;Shmueli et al, 2015;Teigen et al, 2019) or ratio scales (e.g.…”

“…Teigen et al, 2019), interval (e.g. Chaudhry et al, 2019;Keil et al, 2000;Shmueli et al, 2015;Teigen et al, 2019) or ratio scales (e.g. Ayub et al, 2019;Keil et al, 2000;Teigen et al, 2019).…”

“…Chaudhry et al, 2019;Keil et al, 2000;Shmueli et al, 2015;Teigen et al, 2019) or ratio scales (e.g. Ayub et al, 2019;Keil et al, 2000;Teigen et al, 2019). Others ask closed-ended discrete choice questions on an ordinal scale about the scenarios (e.g.…”