Design and application of a transparent and scalable weight‐of‐evidence framework: An example from Wabamun Lake, Alberta, Canada

Abstract

A weight-of-evidence (WOE) framework was developed to evaluate potential effects on the aquatic ecosystem of Wabamun Lake (Alberta, Canada) associated with the release of Bunker "C" oil after a train derailment. The wide variety of stakeholders and interested regulatory agencies made it necessary to develop a consistent and transparent approach to assessing ecological effects on multiple ecosystem components within the lake with the use of a large number of lines of evidence (LOEs). Consequently, a scalable WOE framework was necessary to integrate the findings of 38 different LOEs. A priori and a posteriori weighting factors were applied to each individual LOE, and a combination of numeric and nonnumeric rating systems was used to integrate LOEs into an overall WOE conclusion for 5 different ecosystem components. We provide guidance regarding the development of a WOE framework and emphasize techniques that enhance the application of best professional judgement during the WOE process.

Integrating causation in investigative ecological weight of evidence assessments

“…In other cases, this remains an uncertainty in the results that must be acknowledged and that can possibly result in down-weighting, overall, the strength of conclusions. Application of a posteriori considerations parallels the approaches recommended and applied by some authors (Menzie et al 2006;McDonald et al 2007;SABCS 2010), which allow for "adjustment" of WOE weightings after considering the actual LOE results. Where a posteriori weighting is applied, the rationale needs to be documented and justified.…”

“…Qualitative a priori, direction, and a posteriori weighting considerations are applied to determine the strength of quantitative evidence provided by each endpoint and, once grouped, by LOE. Weighting of different endpoints has been suggested for WOE assessments (Menzie et al 1996;McDonald et al 2007;Suter and Cormier 2011;Hope and Clarkson 2014). Weighting considerations to address specific environmental issues typically fall into 4 categories: 1) strength of association between endpoints and ecological values being protected, 2) representativeness (spatial, temporal, site-specificity), 3) study design and execution (for site-specific investigations), and 4) data quality.…”

“…In the interest of maintaining clarity and transparency, the preferred approach is to standardize weightings according to these considerations a priori, such that they can be applied to endpoints and LOE groups in a systematic and nonbiased manner. In practice, the nature of environmental assessment data is complex, and response combinations not foreseen a priori may occur; however, it is possible to allow for a posteriori judgments (i.e., considering the nature and direction of actual endpoint responses) to fine-tune WOE outcomes where warranted (Menzie et al 1996;McDonald et al 2007;SABCS 2010).…”

“…Weight of evidence assessments provide an established and accepted method for integrating environmental assessment data (e.g., Chapman and Anderson 2005;McDonald et al 2007;Environment Canada and Ontario Ministry of the Environment 2008;Cormier et al 2010;Suter et al 2010;Suter and Cormier 2011;Chapman and Smith 2012;Wolfram et al 2012;Cormier and Suter 2013;ECHA 2014;Guti errez et al 2014;Hope and Clarkson 2014;Khim and Hong 2014;Bonnail et al 2016). Linkov et al (2015, p 4) state "integration of individual lines of evidence is an essential component of environmental assessments.…”

“…Semiquantitative investigative approaches begin in the same way, but then convert the classifications to numerical scores, to which numerical weighting factors are applied, to obtain an overall WOE "score" (e.g., frameworks proposed by Menzie et al 1996;Hope and Clarkson 2014). Hybrid investigative approaches also have applied numerical ratings and weightings systematically, but then the overall WOE score is interpreted qualitatively rather than absolutely (e.g., McDonald et al 2007). Retrospective approaches aimed at determining the causes of detected adverse impacts also can employ all 3 approaches, with a key difference that an adverse impact has already been determined and the next step is to compile the available information to identify the most likely causes to better guide management and mitigation (e.g., environmental epidemiology principles of the US Environmental Protection Agency [USEPA] Causal Analysis/Diagnosis Decision Information System [CAD-DIS]; http://www.epa.gov/caddis/).…”