Development of an adaptive monitoring framework for long‐term programs: An example using indicators of fish health

Abstract

Detecting unwanted changes associated with localized human activities in aquatic ecosystems requires defining the value of an indicator expected at a site in the absence of development. Ideally, adequate and comparable baseline data will be collected at an exposure location before that development, but this is rarely done. Instead, comparisons are made using various designs to overcome the inadequate or missing baseline data. Commonly these comparisons are done over short periods, using information from local reference sites to estimate variability expected at the exposed site. Results of these truncated designs are often evaluated using p values that may have little bearing on ecologically relevant changes. To remedy the reliance of studies on small datasets collected at reference sites, other designs emphasize regional analyses, but these may be insensitive to site-specific changes. Some designs also may forego discussing the consequences of detecting any differences. A new monitoring framework has been proposed to use existing solutions, simplify analysis, and focus on the detection of meaningful changes. It is illustrated here by using data on fish health from a large-scale, long-term program in the Moose River basin in northern Ontario. This framework advocates interpretation of data at multiple scales: within-site, locally, and regionally. The primary focus is on estimating a range from a probability distribution of historical data collected at a specific location where 95% of future observations are predicted to occur. Changes at the exposed site are also compared with historical and contemporary expectations from proximate and regional reference sites. Critical effect sizes also can be derived from regional reference data to evaluate the magnitude of differences observed between any 2 sites. Any unexpected changes inform future monitoring decisions provided by a priori guidance. Adoption of this framework extends the utility of monitoring programs in which commitments to long-term collections have been made, advocates harmonization of studies over time and space, and focuses attention on unusual observations.

Cautions on using the Before-After-Control-Impact design in environmental effects monitoring programs

“…As far as the ramping rate experiment goes, exploring the mechanisms behind the observed biotic responses, if they continue with the inclusion of additional years of monitoring, requires an examination of environmental characteristics such as the flow and temperature regimes, and this is the subject of ongoing analyses. Also still to be considered is if the statistical significance translates into biological relevance for the impacted system (Arciszewski and Munkittrick 2015), but that question is beyond the scope of this paper.…”

Using adaptive processes and adverse outcome pathways to develop meaningful, robust, and actionable environmental monitoring programs

“…Although not ideal, comparisons to reference sites are adequate for interim analyses. These comparisons may also be used to progress through tiers but may rely on an onerous burden of proof, such as a probable effect or alarm level (Arciszewski and Munkittrick ). This initial effects monitoring is commonly done and can provide information to initiate an adaptive program.…”

“…Exceeding a threshold often attracts attention, but as initiators of some further monitoring action, including refinement of questions, thresholds are operationalized as triggers. Elsewhere published CESs can serve as preliminary triggers, but their development and refinement is an important and explicit component of an evolving monitoring program (Arciszewski and Munkittrick 2015). Where refinement is desired, a parametric 95% prediction range is commonly advocated as a trigger and placeholder for change that may be relevant (Munkittrick et al 2009;Arciszewski and Munkittrick 2015).…”

“…Elsewhere published CESs can serve as preliminary triggers, but their development and refinement is an important and explicit component of an evolving monitoring program (Arciszewski and Munkittrick 2015). Where refinement is desired, a parametric 95% prediction range is commonly advocated as a trigger and placeholder for change that may be relevant (Munkittrick et al 2009;Arciszewski and Munkittrick 2015). Virtually any coverage probability can be used but requires acknowledging the accompanying (and expected) error rates.…”

Defining the role of omics in assessing ecosystem health: Perspectives from the Canadian environmental monitoring program

“…Substantial recent work has focused on defining so-called normal ranges for endpoints [60], and this philosophy should, in our opinion, be integrated into omics designs to improve interpretability of molecular (and physiological) indicators. One of the main barriers for inclusion of molecular enpoints in the Canadian EEM program was an inability to define the magnitude of change that would be interpreted as a problem in each species for which the indicator would be used; because of the diversity of receiving environments in Canada, the Canadian EEM program has, so far, used 65 different species of fish [18].…”