Experimental Evaluation of Scale, and Patterns of Systematic Inconsistencies in Google Trends Data

Behnen, Philipp; Keßler, René; Kruse, Felix; Gómez, Jorge Marx; Schoenmakers, Jan; Zerr, Sergej

doi:10.1007/978-3-030-65965-3_25

Cited by 6 publications

(7 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When a group of scientists in Germany investigated the Google Trends indices (this is an index going from 0 to 100 that Google Trends creates for each search term, measuring its relative popularity at a particular time and place) for the terms “dachdecker”, “kurzarbeit”, and “sofa” in Germany, the researchers also noted, as I did (although I looked at the Google Trends graphs and not at the indices), that depending on the time of the search, results could vary greatly. Concerning the words “kurzarbeit” and “dachdecker”, the scientists observed that the relative standard deviation was more than 100% between 15 April and 17 April 2020 for these search terms in Germany (Behnen et al, 2020: 380).…”

Section: Google Trends—an Unplanned Experimentsmentioning

confidence: 99%

Big data, big problems: Why scientists should refrain from using Google Trends

Franzén

2023

Acta Sociologica

View full text Add to dashboard Cite

Google Trends has for over a decade been used by researchers in medicine and the social sciences who want to use information about internet searches to gain new data and insights concerning medical and social issues. A similar tool by the same company, Google Flu Trends, was abolished by Google in 2015 due to serious problems with accuracy; raising larger questions about the quality of the data provided, not only by Google, but by all platforms collecting big data. In this article, I use an unplanned experiment to test the reliability and replicability of Google Trends. The results strongly indicate that scientists in all fields should refrain from using the tool Google Trends when conducting research.

show abstract

Section: Google Trends—an Unplanned Experimentsmentioning

confidence: 99%

Big data, big problems: Why scientists should refrain from using Google Trends

Franzén

2023

Acta Sociologica

View full text Add to dashboard Cite

show abstract

“…Nor did she define the criteria she used for determining what is and is not ‘identical’, nor the exact amount of variation in each attempt. She does cite Behnen et al (2020), who studied the issue in more detail. However, she misses the point they raise which is directly applicable to her own use of GT – ‘Upon our contacting them, Google posited that such deviations may happen, but should only occur for requests with small search volumes and should be marginal’ (Behnen et al, 2020: 376, emphasis added).…”

Section: Are Gt Data Replicable?mentioning

confidence: 99%

“…She does cite Behnen et al (2020), who studied the issue in more detail. However, she misses the point they raise which is directly applicable to her own use of GT – ‘Upon our contacting them, Google posited that such deviations may happen, but should only occur for requests with small search volumes and should be marginal’ (Behnen et al, 2020: 376, emphasis added). Behnen et al also provide an important note to an online discussion (Anonymous, 2019), giving the same answer to this conundrum that I have noted here – sampling.…”

Section: Are Gt Data Replicable?mentioning

confidence: 99%

Of babies, bathwater, and big data: Going beneath the surface of Franzén’s (2023) Google Trends recommendations

Raubenheimer

2023

Acta Sociologica

View full text Add to dashboard Cite

Franzén (2023 : 1) has warned of ‘big problems’ when researchers attempt to use Google Trends (GT) data. The evidence she provides is examined, additional evidence is obtained and analysed, and a new set of conclusions are derived. The anomalies previously encountered are due to a combination of factors, but can be explained by noting that Google samples its data to provide GT results, these data are also scaled, which can exacerbate variation between samples, and Denmark is a small country and Jakob Scharf a low-probability search term, both of which would increase variation in search probabilities provided by GT. When multiple samples are obtained and aggregated (medians are best for low-probability search terms), this variation is controlled, and a stable time series is derived. Researchers should not see GT as an easy source of data, but should do the work required to understand the data, and should use it, and interpret their results, within the limitations inherent in these data. It is important to aggregate multiple samples (preferably with the median for each time point) in order to obtain more stable estimates from GT.

show abstract

“…Although Google may have explanations for the deviations, they are not publicly shared (Behnen et al, 2020). Furthermore, spurious correlations between GT data and phenomena that need to be forecasted are found (Stephens-Davidowitz & Varian, 2014).…”

Section: Google Trends Forecastingmentioning

confidence: 99%

“…Li et al, 2017;Varian, 2014). This complexity is amplified within GT since GT data structurally changes over time (Behnen et al, 2020;Nagao et al, 2019) causing relationships to only hold for a limited period (Zagheni & Weber, 2015). Therefore, our research question is "What impact does model re-specification have on forecasting accuracy when there are time-varying relationships between variables?"…”

Section: Introductionmentioning

confidence: 99%

Google Trends Forecasting: Model re-specification of time-series for enhancing forecasts of youth unemployment

Wijnhoven,

De Bruijn,

Effing

2023

Preprint

View full text Add to dashboard Cite

In the last decades, the forecasting field has been using the surge in big data and advanced computational capabilities. Despite these developments, forecasters continue using traditional forecasting procedures that assume static relationships between phenomena. To address the reality of dynamic relations among phenomena, this study discusses time-variant re-specification methods as part of time-series based forecasts and compares the outcomes with the traditional procedures. This method-comparison is applied to a real-world exercise, the forecasting of Dutch youth unemployment with big data based on Google Trends. For youth unemployment forecasts, our results show 44% more forecasting accuracy by time-varying forecasting models than the traditional static forecasting models. Additionally, this study makes labour market forecasting an accessible endeavour to all organizations by sharing the algorithm for forecasting youth unemployment rates with publicly available data such as Google Trends. Moreover, our study stresses a reconsideration of forecasting methodologies towards model re-specification instead of model recalibration.

show abstract

Experimental Evaluation of Scale, and Patterns of Systematic Inconsistencies in Google Trends Data

Cited by 6 publications

References 17 publications

Big data, big problems: Why scientists should refrain from using Google Trends

Big data, big problems: Why scientists should refrain from using Google Trends

Of babies, bathwater, and big data: Going beneath the surface of Franzén’s (2023) Google Trends recommendations

Google Trends Forecasting: Model re-specification of time-series for enhancing forecasts of youth unemployment

Contact Info

Product

Resources

About