Expert judgements are essential when time and resources are stretched or we face novel dilemmas requiring fast solutions. Good advice can save lives and large sums of money. Typically, experts are defined by their qualifications, track record and experience [1], [2]. The social expectation hypothesis argues that more highly regarded and more experienced experts will give better advice. We asked experts to predict how they will perform, and how their peers will perform, on sets of questions. The results indicate that the way experts regard each other is consistent, but unfortunately, ranks are a poor guide to actual performance. Expert advice will be more accurate if technical decisions routinely use broadly-defined expert groups, structured question protocols and feedback.

Methods for eliciting and aggregating expert judgment are necessary when decision-relevant data are scarce. Such methods have been used for aggregating the judgments of a large, heterogeneous group of forecasters, as well as the multiple judgments produced from an individual forecaster. This paper addresses how multiple related individual forecasts can be used to improve aggregation of probabilities for a binary event across a set of forecasters. We extend previous efforts that use probabilistic incoherence of an individual forecaster's subjective probability judgments to weight and aggregate the judgments of multiple forecasters for the goal of increasing the accuracy of forecasts. With data from two studies, we describe an approach for eliciting extra probability judgments to (i) adjust the judgments of each individual forecaster, and (ii) assign weights to the judgments to aggregate over the entire set of forecasters. We show improvement of up to 30% over the established benchmark of a simple equal-weighted averaging of forecasts. We also describe how this method can be used to remedy the “fifty–fifty blip” that occurs when forecasters use the probability value of 0.5 to represent epistemic uncertainty.

No abstract

Computer-based argument mapping greatly enhances student critical thinking, more than tripling absolute gains made by other methods. I describe the method and my experience as an outsider. Argument mapping often showed precisely how students were erring (for example: confusing helping premises for separate reasons), making it much easier for them to fix their errors.

We investigatedthe perception of causation via the ability to detect conservation violations in simple events. We showed that observers were sensitive to energy conservation violations in free-fall events. Furthermore, observers were sensitive to gradually perturbed energy dynamics in such events. However, they were more sensitive to the effect of decreasing gravity than to that of increasing gravity. Displays with decreasing gravity were the only displays in which the energy profile was dominated by (apparent) potential energy, leading to an asymmetric trajectory.