Large-Scale Web Scraping for Problem Gambling Research: A Case Study of COVID-19 Lockdown Effects in Germany

Smith, Elke; Michalski, Simon; Knauth, Kilian; Kaspar, Kai; Reiter, Nils; Peters, Jan

doi:10.1007/s10899-023-10187-1

Cited by 6 publications

(2 citation statements)

References 60 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Data within each channel were z-standardised to facilitate comparisons between channels with varying levels of engagement. Following earlier work (Smith et al, 2023), we also considered the temporal pattern of user engagement, specifically the frequency of comments over time within the first seven days after video publication (comment latencies). We therefore calculated the interval between the time a video was uploaded and the time a comment was published.…”

Section: User Engagement Measuresmentioning

confidence: 99%

Loot box content is associated with increased user engagement for YouTube gaming videos

Smith,

Poth,

Sohns

et al. 2023

Preprint

View full text Add to dashboard Cite

Video games frequently contain loot boxes, i.e. purchasable randomised virtual in-game items which share structural similarities with gambling. On YouTube©, there are multi-million subscriber video streams focusing on gaming, and many videos also prominently feature content related to loot boxes (e.g. box opening videos). We investigated whether user engagement for YouTube gameplay videos is related to the display of loot boxes, assuming that the gamblification of video games drives user engagement. Using web scraping, we collected various user engagement measures from selected YouTube gaming videos with and without focused display of loot boxes. The user engagement measures were subjected to dimensionality reduction via principal component analysis, and differences in user engagement for loot and non-loot box video content was examined. Confirming our hypothesis (see preregistration at https://osf.io/nh7zr), we found a significant positive effect of loot box content for the first component, reflecting higher active (i.e. commenting behaviour) and passive (i.e. views and likes) user engagement for loot box compared to non-loot box content. Higher user engagement for loot box video content may be linked to the gambling-like properties of loot boxes. Further, our analyses demonstrate that publicly available user data may serve as an early indicator of potential changes in problematic internet use and gambling behaviour.

show abstract

Section: User Engagement Measuresmentioning

confidence: 99%

Loot box content is associated with increased user engagement for YouTube gaming videos

Smith,

Poth,

Sohns

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…As of January 2023, a search of the keyword “web scraping” in Medline yielded 105 records, 95 of which were published starting from 2019. Articles using web scraping in health and social research mostly used information from social media [ 13 , 14 ] (eg, Twitter, Instagram, and TikTok), forums [ 15 , 16 ], business and review websites [ 17 ], and news web pages [ 18 ].…”

Section: Introductionmentioning

confidence: 99%

Generating Contextual Variables From Web-Based Data for Health Research: Tutorial on Web Scraping, Text Mining, and Spatial Overlay Analysis

Galvez-Hernandez,

Gonzalez-Viana,

Gonzalez-de Paz

et al. 2024

JMIR Public Health Surveill

View full text Add to dashboard Cite

Background Contextual variables that capture the characteristics of delimited geographic or jurisdictional areas are vital for health and social research. However, obtaining data sets with contextual-level data can be challenging in the absence of monitoring systems or public census data. Objective We describe and implement an 8-step method that combines web scraping, text mining, and spatial overlay analysis (WeTMS) to transform extensive text data from government websites into analyzable data sets containing contextual data for jurisdictional areas. Methods This tutorial describes the method and provides resources for its application by health and social researchers. We used this method to create data sets of health assets aimed at enhancing older adults’ social connections (eg, activities and resources such as walking groups and senior clubs) across the 374 health jurisdictions in Catalonia from 2015 to 2022. These assets are registered on a web-based government platform by local stakeholders from various health and nonhealth organizations as part of a national public health program. Steps 1 to 3 involved defining the variables of interest, identifying data sources, and using Python to extract information from 50,000 websites linked to the platform. Steps 4 to 6 comprised preprocessing the scraped text, defining new variables to classify health assets based on social connection constructs, analyzing word frequencies in titles and descriptions of the assets, creating topic-specific dictionaries, implementing a rule-based classifier in R, and verifying the results. Steps 7 and 8 integrate the spatial overlay analysis to determine the geographic location of each asset. We conducted a descriptive analysis of the data sets to report the characteristics of the assets identified and the patterns of asset registrations across areas. Results We identified and extracted data from 17,305 websites describing health assets. The titles and descriptions of the activities and resources contained 12,560 and 7301 unique words, respectively. After applying our classifier and spatial analysis algorithm, we generated 2 data sets containing 9546 health assets (5022 activities and 4524 resources) with the potential to enhance social connections among older adults. Stakeholders from 318 health jurisdictions registered identified assets on the platform between July 2015 and December 2022. The agreement rate between the classification algorithm and verified data sets ranged from 62.02% to 99.47% across variables. Leisure and skill development activities were the most prevalent (1844/5022, 36.72%). Leisure and cultural associations, such as social clubs for older adults, were the most common resources (878/4524, 19.41%). Health asset registration varied across areas, ranging between 0 and 263 activities and 0 and 265 resources. Conclusions The sequential use of WeTMS offers a robust method for generating data sets containing contextual-level variables from internet text data. This study can guide health and social researchers in efficiently generating ready-to-analyze data sets containing contextual variables.

show abstract

The impact of COVID-19 pandemic on gambling: A systematic review

Catalano,

Milani,

Franco

et al. 2024

Addictive Behaviors

View full text Add to dashboard Cite

Large-Scale Web Scraping for Problem Gambling Research: A Case Study of COVID-19 Lockdown Effects in Germany

Cited by 6 publications

References 60 publications

Loot box content is associated with increased user engagement for YouTube gaming videos

Loot box content is associated with increased user engagement for YouTube gaming videos

Generating Contextual Variables From Web-Based Data for Health Research: Tutorial on Web Scraping, Text Mining, and Spatial Overlay Analysis

The impact of COVID-19 pandemic on gambling: A systematic review

Contact Info

Product

Resources

About