Infodemic Signal Detection During the COVID-19 Pandemic: Development of a Methodology for Identifying Potential Information Voids in Online Conversations

Abstract

Background The COVID-19 pandemic has been accompanied by an infodemic: excess information, including false or misleading information, in digital and physical environments during an acute public health event. This infodemic is leading to confusion and risk-taking behaviors that can be harmful to health, as well as to mistrust in health authorities and public health responses. The World Health Organization (WHO) is working to develop tools to provide an evidence-based response to the infodemic, enabling prioritization of health response activities. Objective In this work, we aimed to develop a practical, structured approach to identify narratives in public online conversations on social media platforms where concerns or confusion exist or where narratives are gaining traction, thus providing actionable data to help the WHO prioritize its response efforts to address the COVID-19 infodemic. Methods We developed a taxonomy to filter global public conversations in English and French related to COVID-19 on social media into 5 categories with 35 subcategories. The taxonomy and its implementation were validated for retrieval precision and recall, and they were reviewed and adapted as language about the pandemic in online conversations changed over time. The aggregated data for each subcategory were analyzed on a weekly basis by volume, velocity, and presence of questions to detect signals of information voids with potential for confusion or where mis- or disinformation may thrive. A human analyst reviewed and identified potential information voids and sources of confusion, and quantitative data were used to provide insights on emerging narratives, influencers, and public reactions to COVID-19–related topics. Results A COVID-19 public health social listening taxonomy was developed, validated, and applied to filter relevant content for more focused analysis. A weekly analysis of public online conversations since March 23, 2020, enabled quantification of shifting interests in public health–related topics concerning the pandemic, and the analysis demonstrated recurring voids of verified health information. This approach therefore focuses on the detection of infodemic signals to generate actionable insights to rapidly inform decision-making for a more targeted and adaptive response, including risk communication. Conclusions This approach has been successfully applied to identify and analyze infodemic signals, particularly information voids, to inform the COVID-19 pandemic response. More broadly, the results have demonstrated the importance of ongoing monitoring and analysis of public online conversations, as information voids frequently recur and narratives shift over time. The approach is being piloted in individual countries and WHO regions to generate localized insights and actions; meanwhile, a pilot of an artificial intelligence–based social listening platform is using this taxonomy to aggregate and compare online conversations across 20 countries. Beyond the COVID-19 pandemic, the taxonomy and methodology may be adapted for fast deployment in future public health events, and they could form the basis of a routine social listening program for health preparedness and response planning.

Fight Like a Nerdy Girl: The Dear Pandemic Playbook for Combating Health Misinformation

“…Methodological approaches can range from our simple Q&A format to the World Health Organization's highly sophisticated machine learning-driven social listening of large social media platforms. 5 Our methods provide proof-of-concept that even simple efforts to elicit community feedback can reap great benefits if done with intention.…”

“…Cultivating a two-way dialogue, termed “social listening” in infodemiology research, 5 is also considered a best practice in the broader health communication and promotion literatures, 6,7 and is already familiar to and embodied by health promotion professionals. Methodological approaches can range from our simple Q&A format to the World Health Organization’s highly sophisticated machine learning-driven social listening of large social media platforms.…”

A Public Health Research Agenda for Managing Infodemics: Methods and Results of the First WHO Infodemiology Conference

“…The research gaps that were identified have also guided the WHO in the review of the COVID-19 research blueprint [ 42 ] and in the development of partnerships that foster filling of research gaps and for translation of evidence into use by health authorities and other partners [ 28 , 43 ]. The WHO has also applied evidence and infoveillance methods to inform its own work and contribute to the development of metrics for health authorities [ 18 , 44 , 45 ].…”

How to Make Health and Risk Communication on Social Media More “Social” During COVID-19

“…In their battle against misinformation, health organizations should change the pejorative framing of the term "infodemic" into something more neutral so as not to generate resistance or a boomerang effect among certain groups. For example, the WHO can use its AI-powered tool 86 based in its social listening approach 87 to ask subgroups in each country to suggest other names for the information and can consult with those who are hesitant or skeptical regarding how to reframe the information.…”

“…128 The public sphere would function better as a system in times of crisis if the WHO were to interact with various groups in the public on an ongoing basis, like in the WHO recent social listening approach. 86,87 The culture of inclusion should be established not only during crises, but as a systemic continuum for digital crowdsourcing risk communication management.…”