Threats of Bots and Other Bad Actors to Data Quality Following Research Participant Recruitment Through Social Media: Cross-Sectional Questionnaire

Abstract

Background Recruitment of health research participants through social media is becoming more common. In the United States, 80% of adults use at least one social media platform. Social media platforms may allow researchers to reach potential participants efficiently. However, online research methods may be associated with unique threats to sample validity and data integrity. Limited research has described issues of data quality and authenticity associated with the recruitment of health research participants through social media, and sources of low-quality and fraudulent data in this context are poorly understood. Objective The goal of the research was to describe and explain threats to sample validity and data integrity following recruitment of health research participants through social media and summarize recommended strategies to mitigate these threats. Our experience designing and implementing a research study using social media recruitment and online data collection serves as a case study. Methods Using published strategies to preserve data integrity, we recruited participants to complete an online survey through the social media platforms Twitter and Facebook. Participants were to receive $15 upon survey completion. Prior to manually issuing remuneration, we reviewed completed surveys for indicators of fraudulent or low-quality data. Indicators attributable to respondent error were labeled suspicious, while those suggesting misrepresentation were labeled fraudulent. We planned to remove cases with 1 fraudulent indicator or at least 3 suspicious indicators. Results Within 7 hours of survey activation, we received 271 completed surveys. We classified 94.5% (256/271) of cases as fraudulent and 5.5% (15/271) as suspicious. In total, 86.7% (235/271) provided inconsistent responses to verifiable items and 16.2% (44/271) exhibited evidence of bot automation. Of the fraudulent cases, 53.9% (138/256) provided a duplicate or unusual response to one or more open-ended items and 52.0% (133/256) exhibited evidence of inattention. Conclusions Research findings from several disciplines suggest studies in which research participants are recruited through social media are susceptible to data quality issues. Opportunistic individuals who use virtual private servers to fraudulently complete research surveys for profit may contribute to low-quality data. Strategies to preserve data integrity following research participant recruitment through social media are limited. Development and testing of novel strategies to prevent and detect fraud is a research priority.

Impacts of the SARS-CoV-2 Pandemic on Young Adult Colorectal Cancer Survivors

“…To ensure validity and integrity of data and reduce potential fraudulent responses encountered in social media recruitment, participants were first asked questions regarding eligibility to eliminate automated software or ''bots.'' 12 Additional steps to reduce fraudulent responses included prohibition of duplicate emails; removal of respondents whose survey completion was <5 minutes given average completion time of 17 minutes; and removal of respondents reporting ''highly improbable'' medical treatment patterns (e.g., patients who reported stage 1 colon cancer and reported receiving immunotherapy) as reviewed by a medical oncologist (A.B. ).…”

Parent–Child Relationships and the COVID-19 Pandemic: An Exploratory Qualitative Study with Parents in Early, Middle, and Late Adulthood

“…To check for understanding and refine the survey, the survey was piloted with 20 adults prior to data collection. Various strategies were used with the online survey to ensure data quality (e.g., inclusion of open-ended items, raffle incentive vs renumeration for all, embedded directive items; Pozzar et al, 2020). Sample inclusion required the participant to have at least one child.…”

Rapid Deployment of A Community Engagement Study And Educational Trial Via Social Media: Implementation of The UC-COVID Study

“…In prior work using similar recruitment methods, (7,15) identi cation of bots or other sources of invalid records is of particular concern and several methods for such identi cation have been utilized. We importantly identi ed that even with survey platform tools (such as the 'Prevent Ballot Box Stu ng' option), additional attention to records' metadata is critical to identify residual entries submitted from bots.…”

“…However, clearing browser cookies, switching to a different web browser, using a different device, or using a browser in 'incognito' mode would all allow a participant to enter the survey again. As such, we additionally relied on embedded data to identify potential fraudulent entries for records attached to IP addresses that were duplicated in the data greater than four times; three of four instances were suspected to be the result of bots (fraudulent activity) (15) and discarded from the data. In the rst instance, one IP address (geotagged to a location in China) contributed 172 attempted survey entries, none of which progressed in the survey beyond the consent page, that were all submitted within a 24-minute window.…”