Challenges in Recruiting U.S. Farmers for Policy‐Relevant Economic Field Experiments

Abstract

To develop evidence-based agricultural policies, researchers increasingly use insights from economic field experiments. These insights are often limited by the challenges of recruiting large and representative samples of farmers. To improve the effectiveness and cost efficiency of farmer recruitment, researchers should apply the same experimental methods to the recruitment process that they apply to their main research questions. Here we experimentally evaluate ten recruiting strategies in two large-scale, high stakes experiments. We find that monetary incentives and reminders are effective, but costly. Costless strategies, such as prominently citing a well-known institution as the sponsor, had positive but small, effects on recruitment.

Barriers to Using Economic Experiments in Evidence‐Based Agricultural Policymaking

“…The problem of external validity is particularly acute in agricultural experiments because it is rarely possible in practice to recruit a large, representative sample of farmers in an artefactual experiment (see Palm-Forster et al 2016 andWeigel et al 2020). Power analyses indicate that large samples are often needed to detect treatment effects for some types of agri-environmental policies, and recent research has found that many of the agri-economic experiments conducted so far have been underpowered (Palm-Forster et al 2019).…”

“…Testing for heterogeneous treatment effects with adequate power is challenging given the difficulty of recruiting large numbers of farmers for field experiments conducted by academic researchers not embedded within government programs (see Weigel et al 2020). Heterogenous treatment effects can add variance and further reduce the power of a study, but this problem can be mitigated by using techniques such as stratification and block randomization (Duflo, Glennerster, and Kremer 2007).…”

“…Adding more subjects to well-represented strata typically does not make up for underrepresented strata. It may not be feasible for academic researchers to recruit sufficient subjects in all underrepresented strata through conventional recruitment techniques (see Weigel et al 2020). Options for increasing recruitment of underrepresented strata could include expanding the geographic scope of recruitment activities, partnering with private sector businesses and NGOs that serve the targeted community (e.g.…”

“…The most common recruitment methods-in-person recruiting at agricultural conventions/shows and by mail-are inherently prone to generating substantial self-selection issues. Weigel et al 2020 presents a detailed overview of the costs and complexities associated with this type of recruitment.…”

Facebook Ads Manager as a Recruitment Tool for a Health and Safety Survey of Farm Mothers: Pilot Study

“…However, this broad targeting introduces further concerns over multiple biases [13,16]. Although more traditional recruitment methods have been used and evaluated for agricultural populations, often with mixed results, our review of the literature indicates that there has been little data reported on the use of Facebook or other social media platforms for recruitment [17][18][19][20][21]. Therefore, we undertook this pilot study to investigate the effectiveness of targeted advertisements on Facebook as a tool to recruit participants in the agricultural sector, and to assess possible self-selection and geographic biases in this sampling method.…”