Forecast value: prescriptive decision studies

A skill score based on economic value for probability forecasts

“…The approach is a rational way to approach verification for weather variables that directly affect human enterprises, and would probably be less attractive for forecasts of variables such as geopotential heights. Note that the 2×2 cost/loss ratio situation is essentially a toy decision problem which neglects such complications as multiple decision options, multiple relevant meteorological events, related sequential decisions, and different attitudes toward risk (e.g., Wilks, 1997), although a large number of decision problems Skill score based on economic value for probability forecasts can be viewed at least approximately in terms of the simple cost/loss ratio situation. While it might be possible to devise similar scores based on more complex and realistic decision problems, these would either pertain to quite specific user subsets, or increase the dimensionality of the results (e.g., Figures 3 and 5) in ways that would make concise display difficult.…”

“…Conventionally the economic value of forecasts is computed as the difference of expected expenses between some baseline (often climatological) information and the expected expenses given the forecasts under consideration, EE clim -EE f (e.g., Wilks, 1997). In raw form this difference would be unsuitable as a general measure of forecast performance because of its dependence Skill score based on economic value for probability forecasts …”

“…Thus, summarising forecast performance in terms of BS or SS is deficient from the standpoint of forecast users, even if the users understand the structure of these statistics. At the opposite extreme, sophisticated forecast users with access to the joint distribution p(f i | o j ) (or one of its factorisations, for example in the form of a reliability diagram), and knowledge of their own loss functions can compute the expected economic value of the forecasts for their particular decision problems (e.g., Clemen, 1996;Wilks, 1997;Winkler & Murphy, 1985). However, typical forecast users would probably find a tabulation of p(f i | o j ) to be too complex and thus fairly uninformative.…”

“…The cost/loss problem was apparently introduced first by Thompson (1952), and has been studied extensively since that time (e.g., Katz & Murphy, 1997;Murphy, 1977;Murphy & Ehrendorfer, 1987;Thompson & Brier, 1955). While it is the simplest possible quantitative model of optimal decision-making, it nevertheless can approximate many real-world problems to a reasonable degree (Roebber & Bosart, 1996;Wilks, 1997).…”

Analysing the economic value of meteorological information to improve crop risk management decisions in a dynamic context

“…The cost-loss model has been widely used to analyse this kind of management problem (Katz and Murphy, 1997;Cerdá and Quiroga, 2010;Wilks, 1997;Palmer, 2002), and several examples with numerical results have been presented for a farmer's protection in a dynamic framework (Katz, 1993;Meza et al, 2003;Katz and Ehrendorfer, 2006). Analytical results for the optimal policy of the dynamic model for the case of climatological information have been calculated in Cerdá and Quiroga (2011).…”