Concurrently Assessing Survey Mode and Sample Size in Off‐Site Angler Surveys

Abstract

Off‐site angler surveys are commonly administered via two or more survey modes in the form of a mixed‐mode survey. Mixed‐mode surveys allow survey administrators to attain the benefits inherent to different survey modes, reduce total survey error, and control survey cost. However, these benefits can only be simultaneously attained after undertaking sample size planning. Sample size planning is a trade‐off analysis wherein a researcher concurrently assesses survey administration cost, the accuracy and precision of estimates, the magnitude and direction of biases, and variance of the test statistic to determine an optimal sample size. We used data from an off‐site angler survey administered to anglers targeting White Sturgeon Acipenser transmontanus to illustrate a systematic approach to sample size planning. Our survey design included a mixed‐mode design with three survey modes (e‐mail, mail, and telephone) and a two‐phase sampling design that had a first contact and a follow‐up contact with a subsample of nonrespondents. Sample size planning was undertaken in the form of a sensitivity analysis wherein four survey design alternatives were simulated and assessed based on four criteria (i.e., bias, precision, accuracy, and cost). We also incorporated tests for nonresponse bias and survey mode effect. We found that (1) response rates were lower for e‐mail surveys (22%) than for mail surveys and telephone surveys (39–44%); (2) nonresponse bias did not have a substantial effect on survey estimates from the mixed‐mode design; and (3) estimates (total effort and total catch) from the mail and e‐mail survey modes were significantly different, indicating a survey mode effect. The high variability of anglers’ annual catch made survey estimates highly imprecise at lower sample sizes. The level of acceptable error varies for each study. Therefore, a systematic approach to sample size planning is necessary to determine the point where acceptable error is reached while considering multiple survey design alternatives. Received September 25, 2016; accepted April 24, 2017 Published online June 19, 2017

Trends in a recreational fishery for mussels in eastern South Africa, based on postal, telephone and online surveys

“…The difference in fisher avidity between postal respondent and nonrespondent groups could clearly be seen in the data of follow-up telephone surveys, in which postal non-respondents consistently reported lower avidity than postal survey respondents (see Figure 4). This result corresponded with surveys of other fisheries (Barrett et al, 2017;Connelly et al, 2000;Thomson, 2013), where active participants were more likely to respond than those that fished rarely. Nevertheless, the catch per outing (and hence CPUE) reported by avid-and less active fishers was similar, with both groups reporting a full daily bag limit on most occasions.…”

“…Furthermore, online response rates decreased sharply when it was the only survey mode used. Mixed-mode surveys based on a combination of survey modes have been shown to achieve higher response rates (Barrett et al, 2017;De Leeuw, 2005;Fowler et al, 2002;Wallen et al, 2016). Changing to a strategy of online and telephonic follow-up surveys will need considerable preparation, including capturing the email addresses of fishers when they buy permits at post offices; extending geographical coverage more evenly across residential districts; popularising online questionnaires through further development of real-time applications and advertising, and by improving feed-back summaries of results to respondents (Gigliotti and Henderson, 2015).…”

“…For example, data obtained from self-administered questionnaires may be affected by declining response rates, with non-respondent groups differing systematically from those that responded to surveys (De Leeuw and De Heer, 2002;Groves, 2006;Wallen et al, 2016). Follow-up surveys are then required to correct for non-respondents (Barrett et al, 2017;Connelly et al, 2000;McClanahan and Hansen, 2005;Tarrant et al, 1993;Van der Hammen et al, 2016). Mixed-mode designs that use a combination of survey types improve response rates (De Leeuw, 2005), but may introduce mode bias by sampling demographically different groups (Lesser et al, 2016;Wallen et al, 2016;Zarauz et al, 2015).…”

Using catch orientations to identify clusters of North Carolina trout anglers

“…Therefore, we used the online Qualtrics survey platform for recruitment and data collection (Qualtrics, Provo, Utah). Single‐mode online surveys in natural resource contexts generally result in lower response rates and could bias demographic characteristics of respondents (Gigliotti and Dietsch 2014; Barrett et al 2017; Leone et al 2022). However, there is evidence that survey mode shows weak variation in angling‐specific characteristics of respondents (e.g., motivations, preferences, behaviors; Graefe et al 2011; Wallen et al 2016; Henderson and Gigliotti 2018).…”

A comparison of mixed‐mode survey designs for collecting deer and fall turkey harvest data in North Dakota

“…Recent research related to hunter and angler survey methodology has focused on response rates and demographic differences between internet and mail survey respondents (Gigliotti 2011, Lesser et al 2011, Laborde et al 2014, Barrett et al 2017). Overall, there has been a downward trend in survey response rate (Kreuter 2013), including those from mail surveys pertaining to natural resources (Stedman et al 2019).…”