When can we ignore the problem of imperfect detection in comparative studies?

Abstract

Summary1. Numbers of individuals or species are often recorded to test for variations in abundance or richness between treatments, habitat types, ecosystem management types, experimental treatments, time periods, etc. However, a difference in mean detectability among treatments is likely to lead to the erroneous conclusion that mean abundance differs among treatments. No guidelines exist to determine the maximum acceptable difference in detectability. 2. In this study, we simulated count data with imperfect detectability for two treatments with identical mean abundance (N) and number of plots (n plots ) but different mean detectability (p). We then estimated the risk of erroneously concluding that N differed between treatments because the difference in p was ignored. The magnitude of the risk depended on p, N and n plots . 3. Our simulations showed that even small differences in p can dramatically increase this risk. A detectability difference as small as 4-8% can lead to a 50-90% risk of erroneously concluding that a significant difference in N exists among treatments with identical N = 50 and n plots = 50. Yet, differences in p of this magnitude among treatments or along gradients are commonplace in ecological studies. 4. Fortunately, simple methods of accounting for imperfect detectability prove effective at removing detectability difference between treatments. 5. Considering the high sensitivity of statistical tests to detectability differences among treatments, we conclude that accounting for detectability by setting up a replicated design, applied to at least part of the design scheme and analysing data with appropriate statistical tools, is always worthwhile when comparing count data (abundance, richness).

Evaluating estimators of species richness: the importance of considering statistical error rates

“…When the assumptions of statistical tests are violated, inflated type‐I error rates are expected (Osborne & Waters ; Archaux, Henry & Gimenez ). This has been demonstrated for many common analyses, such as modelling patterns in abundance with mark–recapture analysis (Archaux, Henry & Gimenez ) or Poison GLMs (White & Bennets ). To our knowledge, this is the first study that has evaluated the effect of common assumption violations on type‐I error rates for species richness estimators.…”

“…This can be achieved with empirical and simulation studies as we have demonstrated here and others have demonstrated in the ecological literature (Zurell et al . ; Archaux, Henry & Gimenez ). The practice of simulating the experimental process can reveal the inference consequences of estimator performance when simple measures of bias and precision cannot.…”

Imperfect detection and the determination of environmental flows for fish: challenges, implications and solutions

“…Hangsleben, Allen & Gwinn, ; see Appendix S1 in Supporting Information for a stylised example of how variable p can impact inference about N ). Furthermore, the literature suggests that the probability of spurious conclusions about fish abundance can increase dramatically with only small variations in p (Archaux, Henry & Gimenez, ; Hangsleben et al ., ). For example, Hangsleben et al .…”

Bird-monitoring in Europe – a first overview of practices, motivations and aims

“…Further, monitoring could be stratified to optimize resource allocation between independent samples (i.e., sites), and employ random (or systematic) sampling to secure an unbiased spatial coverage. Importantly, detection probability needs to be accounted for since even low differences in detection probability between site or years can induce spurious conclusions (Archaux et al 2011). It means that repetitive sampling of the same sites within a year should be the rule.…”