Methodological Considerations for Survey-Based Research During Emergencies and Public Health Crises: Improving the Quality of Evidence and Communication

Abstract

The novel coronavirus (COVID-19) outbreak has resulted in a massive amount of global research on the social and human dimensions of the disease. Between academic researchers, governments, and polling firms, thousands of survey projects have been launched globally, tracking aspects like public opinion, social impacts, and drivers of disease transmission and mitigation. This deluge of research has created numerous potential risks and problems, including methodological concerns, duplication of efforts, and inappropriate selection and application of social science research techniques. Such concerns are more acute when projects are launched under the auspices of quick response, time-pressured conditions–and are magnified when such research is often intended for rapid public and policy-maker consumption, given the massive public importance of the topic.

Prospective sampling bias in COVID-19 recruitment methods: experimental evidence from a national randomized survey testing recruitment materials

“…Postcards requesting respondents' views on COVID-19 were sent to two-thirds of the sample, while the rest were asked for their views on 'healthcare in Canada. ' 2 Postcard treatment groups (COVID vs. non-COVID) were randomized, again following the same stratifications mentioned above. Both postcard designs were consistent (bilingual, including both university and funder logos), varying only wording and image used, and can be reviewed as Figs.…”

“…The COVID-19 crisis has led to a wave of survey-based research around the world, albeit sometimes of suspect quality [1]. Well-designed survey research in COVID-19 can help identify social impacts, measure attitudes, and document the ways respondents are adapting [2], which is critical to understanding public behaviors during the pandemic [3,4]. Moreover, carefully designed social research can help to inform policy and response design [5] through providing real-time evidence about onthe-ground conditions and the effectiveness of various interventions.…”

“…The value of such research, however, can be seriously limited by methodological errors and biases [6,7]. Investigations into COVID-19 survey research has already demonstrated the influence, for example, of biases introduced at the level of item design (e.g., how questions or prompts are formulated), such as the way that social desirability bias and research desirability bias can affect the responses that respondents offer in ways that undermine data quality regarding public behaviors [8][9][10].…”

“…. in the Supplementary Materials, which demonstrates the expected number of households sampled in each province vs. the actual number sampled, as well as a percentage of expected (> 1 represents oversampling; < 1 represents under-sampling) 2. The uneven split reflects a compromise between this particular investigation and the overall objectives of the survey project (which tracks attitudes on a wide variety of topics).…”

How well do covariates perform when adjusting for sampling bias in online COVID-19 research? Insights from multiverse analyses

“…In many research areas, the gold standard for recruiting participants is to use probability-based sampling to draw representative inferences for a given population [ 1 – 3 ]. Unfortunately, such efforts are often costly or unfeasible.…”