Demystifying Evidence‐Based Policy Analysis by Revealing Hidden Value‐Laden Constraints

Abstract

Consider any choice that affects some social policy. A decision that considers evidence will, at its heart, contain some kind of explicit or implicit "because" statement: "We are doing X because the evidence says Y." But can evidence ever truly speak for itself, in the sense of being reducible to objective utterances that are either correct or in need of correction? Before answering, consider what you'd prefer. Would you rather receive evidence that was free of any value judgments imposed by human actors, that was laden with value judgments that you agree with, or laden with value judgments that you disagree with? The central assertion of this essay is that, throughout policy analysis but especially in assessments of the costs and benefits of regulating versus encouraging new technologies (cost-benefit analysis, or CBA), the first possibility above is a mirage, and the second and third are self-contradictory. Instead, we are overwhelmingly confronted with a fourth possibility: we receive evidence that appears to be (or is deliberately touted as) value neutral but is suffused with hidden value judgments. In the second part of this essay, entitled "A Guided Tour through Inevitable Value Judgments," I identify in a systematic way approximately sixty-five value judgments that are routinely (in some cases, invariably) made in CBA, but that are kept hidden. For each judgment, I discuss its genesis as it is most commonly invoked in CBA, explain how it is hidden in plain sight, and offer one or more value judgments that could be made instead of or in addition to the conventional one. The alternative judgments highlight the width of the spectrum of reasonable conclusions an analyst could reach merely by substituting other judgments for the ones currently embedded. Bringing hidden value judgments to light is doubly valuable. First, it allows discussion to ensue on a level playing field; instead of conclusory statements about what the evidence says, transparency permits statements taking the form of "when channeled through these value judgments, the evidence says this." Perhaps more importantly, transparency about value judgments permits participants in the discussions to offer interpretations of evidence contingent on there being different value judgments chosen at one or more points in the analysis, interpretations that may suggest that alternative course(s) of action are preferable to the one being championed.

Making Policies about Emerging Technologies

“…He breaks down any process for policy analysis into a series of ten phases, from picking policy “targets” to evaluating the results, and within each phase, he identifies an assortment of the decisions that policy analysts must make and within which value judgments are inevitably hidden. Finkel concludes that if CBA is properly done—“carried out with greater awareness of the ethical choices that must be made—and that can be made in different ways than dogma insists”—then it is “a tool worthy of the weighty uses we have been waiting to apply it to.” In short, while Mannix uses a discussion of CBA to argue that government should not be in the permission‐granting business, Finkel argues that government has been too lax and should use CBA, improved in ways he recommends, to be a much stronger gatekeeper.…”

A critique of Occupational Safety and Health Administration's halfmask respirator assigned protection factor

“…Issues of interindividual variability and uncertainty are inherent in risk assessment and risk management decisions. 10,11 Hattis and Anderson proposed that "in formulating a social objective relating to the equity (fairness) of a [health] risk distribution, the risk manager needs to decide what fraction of the population (Y) must be kept below one or more X levels of individual risk with what level of confidence (Z) . .…”

The Pebble Remains in the Master's Hand: Two Careers Spent Learning (Still) from John Evans

“…In the limiting case, where one assumption predicts substantial risk and an alternative predicts zero risk, the composite uncertainty distribution is completely determined by the values given to p and (1‐ p ), the weights assigned to each of the two incompatible states of nature—which means that the views of one expert can have more influence on a downstream risk management decision than a data set or a test result might have. And there is no way to avoid subjectivity in the assignment of model weights: starting from the premise that assumptions that are controversial should be given equal weights unless we can justify more precise parsing is itself a very value‐laden and restrictive form of weighting (Finkel, 2018, p. S23).…”