Sampling design begets conclusions: the statistical basis for detection of injury to and recovery of shoreline communities after the 'Exxon Valdez' oil spill

Abstract

The joint effect of multiple initial decisions made about sampling design in evaluation of environmental impacts using observational field assessments influences the ability to detect and accurately estimate responses. The design can dictate in advance whether the study can identify even large impacts that truly exist. Following the 'Exxon Valdez' oil spill in 1989, 4 separate studies of effects of the spill on the intertidal biota were conducted. Studies overlapped sufficiently in geographic area, shoreline habitat, and biological response variables to permit contrasts showing how the aggregate of multiple design decisions led to differences in conclusions. The SEP (Shoreline Ecology Program) supported by Exxon and the CHIA (Coastal Habitat Injury Assessment) funded by the Exxon Valdez Oil Spill Trustee Council shared a core approach of establishing a stratified random design of site selection. The Exxon-supported GOA (Gulf of Alaska) study and the NOAA (National Oceanographic and Atmospheric Administration) Hazmat (Hazardous Materials) study both chose to employ subjective choices of fixed sites. Despite many common goals, these 4 studies differed greatly in: (1) sampling effort (area covered per sample quadrat, sample replication within sites, numbers of study sites per category, numbers of samplings, and total areas sampled) and sampling design (philosophy of targeting sampling effort, complete randomization versus matched pair designs, sampling frame, treatment of habitat heterogeneity within sites, interspersion of sites, and control of shoreline treatment and oiling intensity); (2) analytical methodology (analysis of covariance versus paired designs, treatment of subsamples as replicates in F-ratios, logic of inferring recovery, and power calculations); and (3) choice of biological response variables (taxonomic level of analysis, aggregating versus splitting separate communities, and scope of communities and habitats examined). The CHIA and NOAA Hazmat studies of epibiotic responses in sheltered rocky shores of Prince William Sound made several decisions to enhance detection power and produced similar conclusions about large reductions in total biotic cover of intertidal space, Fucus cover, mussel abundance, abundance of the limpet Tectura persona and a balanoid barnacle, and increases in open space and abundance of an opportunistic barnacle, Chthamalus dalli. In contrast, the SEP study of this same habitat and geographic region adopted design choices resulting in lower power of detection in 12 (vs CHIA and vs NOAA Hazmat) of 15 separate decisions (with one tie in each contrast). Accordingly, the SEP study was able to detect declines only in Fucus cover and occasionally in total limpet abundance but not in total epifaunal or mussel or balanoid barnacle abundance and, unlike the results of the other 2 studies, most of the taxa analyzed showed apparent increases rather than decreases from oiling and shoreline treatment. The more powerful GOA and CHIA studies of impacts of oiling in the Gulf of Ala...

Delayed effects on growth and marine survival of pink salmon Oncorhynchus gorbuscha after exposure to crude oil during embryonic development

“…This is consistent with their failure to distinguish between homing and straying salmon in their study streams, and straying salmon can account for > 25% of the pink salmon in some streams (Sharp et al 1994). Deficiencies in the statistical power of Exxonsponsored sampling designs have been reported elsewhere (Peterson et al 2001).…”

A synthesis of variability in nearshore Alaskan marine populations

“…Sensitivity is presumed to be lower in high-energy environments, where oil or other pollutants are more likely to be removed by wave action and where vertebrate consumers are less able to forage (Teal and Howarth 1984;National Research Council 1986). However, sensitivity is little more than an informed guess (Peterson et al 2001). Ideally, necessarily limited resources should not be squandered either by devoting extensive effort to sampling abundant habitats with low sensitivity or by over sampling rare but sensitive habitats in hopes of detecting small but biologically unimportant differences (Peterson et al 2001).…”

“…However, sensitivity is little more than an informed guess (Peterson et al 2001). Ideally, necessarily limited resources should not be squandered either by devoting extensive effort to sampling abundant habitats with low sensitivity or by over sampling rare but sensitive habitats in hopes of detecting small but biologically unimportant differences (Peterson et al 2001). Because of the lack of differences among substrates observed in this study, the optimal sampling design for a monitoring program in the Gulf of Alaska to detect impacts of future oil spills may be to sample habitats in proportion to their abundance.…”

“…Distinguishing human-induced effects from natural variability is a difficult and challenging task (Thrush et al 1994;Paine et al 1996;Wiens 1996;Marsh 2001;Peterson et al 2001). Designing a monitoring program requires, in essence, the ability to see into the future, because the data to be collected will be analyzed to examine the consequences of activities in the future.…”

Assessing impacts of dredge spoil disposal using equivalence tests: implications of a precautionary (proof of safety) approach

“…This can be manifest in calls to emphasize estimation (especially using confidence intervals) rather than testing (Gardner & Altman 1989, Hoenig & Heisey 2001. Indeed, Peterson et al (2001) noted a move in ecological studies away from testing hypotheses toward estimation of the magnitude of effects. This approach permits some freedom for interpretation, and is always useful.…”