Survey Fraud and the Integrity of Web-Based Survey Research

Levi, Ronli; Ridberg, Ronit; Akers, Melissa; Seligman, Hilary K.

doi:10.1177/08901171211037531

Cited by 34 publications

(22 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Fraudulent surveys include duplicate responses from the same participant using different email addresses, providing response patterns, or reaccessing the survey with false responses after being deemed ineligible in the first attempt. To minimize these risks, we implemented a series of combined red flags based on previous literature [ 39 , 40 , 48 , 49 ] and our own team’s experience. Once data collection for the baseline (T2) survey was completed, we downloaded all the data from Qualtrics.…”

Section: Methodsmentioning

confidence: 99%

The Influence of Social Media Interactions and Behaviors on Depressive Symptoms Among Sexual and Gender Minority Young Adults in the United States: Protocol for a Mixed Methods Longitudinal Study

Escobar-Viera¹,

Coulter²,

Friedman³

et al. 2023

JMIR Res Protoc

View full text Add to dashboard Cite

Background Sexual and gender minority (SGM; ie, lesbian, gay, bisexual, transgender, and otherwise queer) young adults experience disparities in depression and other internalizing psychopathology. Although social media use is widespread and SGM people have more social media accounts and are more socially active on them than non-SGM individuals, few studies have examined the impact of social media on depression in this group. Objective The PRIDE iM study will be the first longitudinal, mixed methods research conducted to determine the impact of social media interactions and behaviors as pathways to depressive symptoms among SGM young adults living in the United States. Methods PRIDE iM uses a bookends variation of the longitudinal sequential mixed methods design. Participants will be recruited nationally from social media. First, between July 2019 and February 2020, we conducted a qualitative phase (T1) comprising web-based individual interviews (N=58) to inform the building and content of the quantitative survey. Second, from February 2022 to September 2022, we will conduct a series of web-based surveys (N=1000 at baseline) with 4 data points (T2-T5), each one collected every 6 to 8 weeks. Third, from October 2022 to December 2022, we will conduct a second qualitative phase (T6) of web-based interviews using outcome trajectories found in the longitudinal survey analyses to purposively sample survey participants and conduct web-based interviews to contextualize and explain survey findings. Qualitative data from T1 and T6 will be analyzed using a reflexive thematic analysis approach. As we sought to capture change over time in the association between the main predictors (ie, social media interactions and behaviors) and depressive symptoms, we propose analyzing T2 to T5 data using latent growth models with a structural equation modeling framework. Data integration at the method, interpretation, and reporting levels will be achieved through building and connecting and the use of a staged approach and joint displays, respectively. At all stages, we will assess the fit of data integration as recommended by the principles of best practice for mixed methods research in psychology. Results Data collection will be completed by December 2022. Qualitative data analyses will be completed by March 2023, and quantitative analyses of the primary outcome of interest will be completed by June 2023. Conclusions PRIDE iM will confirm, reject, or uncover the presence of potential relationships between social media interactions and behaviors and depressive symptoms among SGM people. This study represents fundamental groundwork to develop social media–based interventions that target modifiable interactions and behaviors that are most likely to influence mental health outcomes, thus seizing the opportunity to merge the popularity of this medium among SGM people with evidence-based approaches. International Registered Report Identifier (IRRID) DERR1-10.2196/43627

show abstract

Section: Methodsmentioning

confidence: 99%

The Influence of Social Media Interactions and Behaviors on Depressive Symptoms Among Sexual and Gender Minority Young Adults in the United States: Protocol for a Mixed Methods Longitudinal Study

Escobar-Viera¹,

Coulter²,

Friedman³

et al. 2023

JMIR Res Protoc

View full text Add to dashboard Cite

show abstract

“…As part of routine, monthly quality control procedures, we cross-checked all enrolled participants with WIC agencies to confirm WIC participation (a criteria for inclusion). We identified numerous ineligible participants likely created by bots hacking the survey, which the authors describe in detail elsewhere [ 15 ]; all false survey responses ( n = 228) were identified and removed.…”

Section: Methodsmentioning

confidence: 99%

Additional Fruit and Vegetable Vouchers for Pregnant WIC Clients: An Equity-Focused Strategy to Improve Food Security and Diet Quality

et al. 2022

Self Cite

View full text Add to dashboard Cite

Women with low household income and from racial/ethnic minority groups are at elevated risk of food insecurity. Food insecurity during pregnancy is associated with overall less healthy diets, lower intake of the pregnancy-supportive nutrients iron and folate, and significant variations in diet across the course of a month. The goal of this study was to explore the impact of an ongoing $40/month supplement for fruits and vegetables (F&Vs) provided to pregnant people enrolled in the Special Supplemental Nutrition Program for Women and Children (WIC). Our primary outcome was food insecurity using the USDA 6-item survey, and our secondary outcome was dietary intake of F&Vs based on the 10-item Dietary Screener Questionnaire. Participants in intervention and comparison counties completed surveys at enrollment and approximately three months later (n = 609). Mean ± SD food insecurity at baseline was 3.67 ± 2.79 and 3.47 ± 2.73 in the intervention and comparison groups, respectively, and the adjusted between-group change from baseline to follow-up in food insecurity was 0.05 [95% CI: −0.35, 0.44] (p > 0.05). F&V intake (in cup equivalents) was 2.56 ± 0.95 and 2.51 ± 0.89 at baseline in the two groups, and the adjusted mean between-group difference in changes from baseline was −0.06 [−0.23, 0.11] (p > 0.05). Recruitment and data collection for this study coincided with the most intensive of America’s COVID relief efforts. Our results may indicate that small increases in highly targeted food resources make less of a difference in the context of larger, more general resources being provided to individuals and households in need.

show abstract

“…In other instances, researchers who described experiencing an overwhelming number of fraudulent respondents proposed several data-based screening tools, which may be used to flag respondents in existing data (Griffin et al, 2022; Pozzar et al, 2020; Xu et al, 2022). Common data-based screening tools described in the literature include (a) evaluating qualitative responses for duplicate phrasing or illogical answers, (b) identifying discrepancies to verifiable items, and (c) identifying identical response patterns to quantitative surveys (Griffin et al, 2022; Levi et al, 2021; Pozzar et al, 2020; Simone, 2019). Given that bots have the propensity to improve their capacity to pass many question- and data-based screening tools due to algorithmic learning, we considered using common behavioral and cognitive psychological tasks (e.g., anagrams) as an avenue to examine the efficacy of task-based bot screening tools.…”

Section: The Present Studymentioning

confidence: 99%

“…Notably, most bot detection research has been conducted in response to an unexpected encounter with bots, and thus no known study has used experimental methods to investigate the prospective effectiveness of distinct bot screening tools. All known existing contributions to this line of work are case studies that detail how the authors mitigated threats to data integrity when bots attacked their online surveys (Griffin et al, 2022; Levi et al, 2021; Pozzar et al, 2020; Simone, 2019; Storozuk et al, 2020). By using an experimental approach that replicates the features of data collection in extant case studies that report bot intrusions (i.e., health- and social-focused research, posting survey links on social media) and only varying the incentive amount offered, the present study may uncover how incentives affect the proportion of bots detected in online survey research and how incentives affect the effectiveness of bot screening tools.…”

Section: The Present Studymentioning

confidence: 99%

A quasi-experimental study examining the efficacy of multimodal bot screening tools and recommendations to preserve data integrity in online psychological research.

Simone

Cascalheira

Pierce³

2024

American Psychologist

View full text Add to dashboard Cite

Bots are automated software programs that pose an ongoing threat to psychological research by invading online research studies and their increasing sophistication over time. Despite this growing concern, research in this area has been limited to bot detection in existing data sets following an unexpected encounter with bots. The present three-condition, quasi-experimental study aimed to address this gap in the literature by examining the efficacy of three types of bot screening tools across three incentive conditions ($0, $1, and $5). Data were collected from 444 respondents via Twitter advertisements between July and September 2021. The efficacy of five task-based (i.e., anagrams, visual search), question-based (i.e., attention checks, ReCAPTCHA), and data-based (i.e., consistency, metadata) tools was examined with Bonferroni-adjusted univariate and multivariate logistic regression analyses. In general, study results suggest that bot screening tools function similarly for participants recruited across incentive conditions. Moreover, the present analyses revealed heterogeneity in the efficacy of bot screening tool subtypes. Notably, the present results suggest that the least effective bot screening tools were among the most commonly used tools in existing literature (e.g., ReCAPTCHA). In sum, the study findings revealed highly effective and highly ineffective bot screening tools. Study design and data integrity recommendations for researchers are provided.

show abstract

Survey Fraud and the Integrity of Web-Based Survey Research

Cited by 34 publications

References 6 publications

The Influence of Social Media Interactions and Behaviors on Depressive Symptoms Among Sexual and Gender Minority Young Adults in the United States: Protocol for a Mixed Methods Longitudinal Study

The Influence of Social Media Interactions and Behaviors on Depressive Symptoms Among Sexual and Gender Minority Young Adults in the United States: Protocol for a Mixed Methods Longitudinal Study

Additional Fruit and Vegetable Vouchers for Pregnant WIC Clients: An Equity-Focused Strategy to Improve Food Security and Diet Quality

A quasi-experimental study examining the efficacy of multimodal bot screening tools and recommendations to preserve data integrity in online psychological research.

Contact Info

Product

Resources

About