Need of care in interpreting Google Trends-based COVID-19 infodemiological study results: potential risk of false-positivity

Abstract

Background Google Trends (GT) is being used as an epidemiological tool to study coronavirus disease (COVID-19) by identifying keywords in search trends that are predictive for the COVID-19 epidemiological burden. However, many of the earlier GT-based studies include potential statistical fallacies by measuring the correlation between non-stationary time sequences without adjusting for multiple comparisons or the confounding of media coverage, leading to concerns about the increased risk of obtaining false-positive results. In this study, we aimed to apply statistically more favorable methods to validate the earlier GT-based COVID-19 study results. Methods We extracted the relative GT search volume for keywords associated with COVID-19 symptoms, and evaluated their Granger-causality to weekly COVID-19 positivity in eight English-speaking countries and Japan. In addition, the impact of media coverage on keywords with significant Granger-causality was further evaluated using Japanese regional data. Results Our Granger causality-based approach largely decreased (by up to approximately one-third) the number of keywords identified as having a significant temporal relationship with the COVID-19 trend when compared to those identified by Pearson or Spearman’s rank correlation-based approach. “Sense of smell” and “loss of smell” were the most reliable GT keywords across all the evaluated countries; however, when adjusted with their media coverage, these keyword trends did not Granger-cause the COVID-19 positivity trends (in Japan). Conclusions Our results suggest that some of the search keywords reported as candidate predictive measures in earlier GT-based COVID-19 studies may potentially be unreliable; therefore, caution is necessary when interpreting published GT-based study results.

Analyzing Citizens’ and Health Care Professionals’ Searches for Smell/Taste Disorders and Coronavirus in Finland During the COVID-19 Pandemic: Infodemiological Approach Using Database Logs

“…Besides, genuine interest on self-symptoms before they become broadly known to the general public may have faded [ 17 ]. Prior studies have found that Google searches for smell and taste loss varied between countries [ 27 ] but remained at a higher level after the beginning of the pandemic [ 27 , 28 ]. In our study, searches for smell and taste disorders showed visually better prediction among citizens than HCPs, possibly indicating that loss of smell or taste may have been the only concerning symptom of mild COVID-19 cases.…”

Influence of Mass Media on Italian Web Users During the COVID-19 Pandemic: Infodemiological Analysis

“…Although this influence may jeopardize the adoption of Google Trends in many research areas, it remains suitable for conducting infodemiological investigations; in fact, if certain conditions are met (ie, high internet penetration, high Google use, and data set stability), the relative search volumes (RSVs) on Google Trends faithfully reflect the interests of a population [ 25 ]. Moreover, Sato et al [ 29 ] proposed a new method to evaluate, quantify, and eliminate the media disturbance in data sets.…”

A New Infodemiological Approach Through Google Trends: Longitudinal Analysis of COVID-19 Scientific and Infodemic Names in Italy

“…To achieve our goal, we used Google Trends, an online tool developed by Google that allows the user to monitor netizens' web interests in specific keywords, queries, and topics. Specifically, Google Trends has been used successfully in several infodemiological studies; however, its results need to be interpreted cautiously and examined through appropriate methodologies 18,19 . In this regard, this paper also has two secondary purposes: the first is to provide a new procedure to improve the sensitivity and accuracy of Google Trends datasets, while the second is to provide a keyword collection methodology suitable for representing the vast majority of COVID-19-related queries.…”