A Tale of Two Diverse Qualtrics Samples: Information for Online Survey Researchers

Abstract

BackgroundThere is often a lack of transparency in research using online panels related to recruitment methods and sample derivation. The purpose of this study was to describe the recruitment and participation of respondents from two disparate surveys derived from the same online research panel using quota sampling. MethodsA commercial survey sampling and administration company, Qualtrics, was contracted to recruit participants and implement two internet-based surveys. The first survey targeted adults aged 50-75 years old and used sampling quotas to obtain diversity with respect to household income and race/ethnicity. The second focused on women aged 18-49 years and utilized quota sampling to achieve a geographically balanced sample. ResultsA racially and economically diverse sample of older adults (n=419) and a geographically diverse sample of younger women (n=530) were acquired relatively quickly (within 12 and 4 days, respectively). With exception of the highest income level, quotas were implemented as requested. Recruitment of older adults took longer (vs. younger female adults). Although survey completion rates were reasonable in both studies, there were inconsistencies in the proportion of incomplete survey responses and quality fails. ConclusionsCancer researchersand researchers in generalshould consider ways to leverage the use of online panels for future studies. To optimize novel and innovative strategies, researchers should proactively ask questions about panels and carefully consider the strengths and drawbacks of online survey features including quota sampling and forced response. ImpactResults provide practical insights for cancer researchers developing future online surveys and recruitment protocols.

Association Between Public Knowledge About COVID-19, Trust in Information Sources, and Adherence to Social Distancing: Cross-Sectional Survey

“…Data were collected via a cross-sectional national survey. An independent company that specializes in national data collection, Qualtrics Panels, implemented the recruitment procedures [ 15 , 16 ]. Individuals received an email invitation to the study if they preregistered for Qualtrics Panels and completed a baseline survey.…”

Impact of Employment, Essential Work, and Risk Factors on Food Access during the COVID-19 Pandemic in New York State

“…The survey asked about COVID-19 impacts, food access, determinants of health, perceptions of the response, and individual and household characteristics. Respondents that completed the survey faster than half the median time were automatically excluded for poor data quality [ 35 , 36 ]. Each completed survey was reviewed for poor data quality due to straight-lined responses and non-sense answers following Qualtrics recommended data cleaning practices [ 35 , 36 ].…”

“…Respondents that completed the survey faster than half the median time were automatically excluded for poor data quality [ 35 , 36 ]. Each completed survey was reviewed for poor data quality due to straight-lined responses and non-sense answers following Qualtrics recommended data cleaning practices [ 35 , 36 ]. The D’Youville College Institutional Review Board reviewed and approved this research as Exempt prior to commencing data collection.…”

COVID-19: Public Compliance with and Public Support for Stay-at-Home Mitigation Strategies

“…To evaluate public compliance with and support for recommended COVID-19 mitigation strategies, we collected cross-sectional surveys of nationally representative respondents using demographic quota sampling. 4 Surveys were administered to an online respondent panel by Qualtrics, LLC (Provo, Utah, and Seattle, Washington, US), a commercial survey company with a network of participant pools consisting of hundreds of suppliers. Recruitment methodologies include digital advertisements and promotions, word of mouth and membership referrals, social networks, TV and radio advertisements, and offline mail-based approaches.…”