Background The COVID-19 pandemic has been accompanied by an infodemic: excess information, including false or misleading information, in digital and physical environments during an acute public health event. This infodemic is leading to confusion and risk-taking behaviors that can be harmful to health, as well as to mistrust in health authorities and public health responses. The World Health Organization (WHO) is working to develop tools to provide an evidence-based response to the infodemic, enabling prioritization of health response activities. Objective In this work, we aimed to develop a practical, structured approach to identify narratives in public online conversations on social media platforms where concerns or confusion exist or where narratives are gaining traction, thus providing actionable data to help the WHO prioritize its response efforts to address the COVID-19 infodemic. Methods We developed a taxonomy to filter global public conversations in English and French related to COVID-19 on social media into 5 categories with 35 subcategories. The taxonomy and its implementation were validated for retrieval precision and recall, and they were reviewed and adapted as language about the pandemic in online conversations changed over time. The aggregated data for each subcategory were analyzed on a weekly basis by volume, velocity, and presence of questions to detect signals of information voids with potential for confusion or where mis- or disinformation may thrive. A human analyst reviewed and identified potential information voids and sources of confusion, and quantitative data were used to provide insights on emerging narratives, influencers, and public reactions to COVID-19–related topics. Results A COVID-19 public health social listening taxonomy was developed, validated, and applied to filter relevant content for more focused analysis. A weekly analysis of public online conversations since March 23, 2020, enabled quantification of shifting interests in public health–related topics concerning the pandemic, and the analysis demonstrated recurring voids of verified health information. This approach therefore focuses on the detection of infodemic signals to generate actionable insights to rapidly inform decision-making for a more targeted and adaptive response, including risk communication. Conclusions This approach has been successfully applied to identify and analyze infodemic signals, particularly information voids, to inform the COVID-19 pandemic response. More broadly, the results have demonstrated the importance of ongoing monitoring and analysis of public online conversations, as information voids frequently recur and narratives shift over time. The approach is being piloted in individual countries and WHO regions to generate localized insights and actions; meanwhile, a pilot of an artificial intelligence–based social listening platform is using this taxonomy to aggregate and compare online conversations across 20 countries. Beyond the COVID-19 pandemic, the taxonomy and methodology may be adapted for fast deployment in future public health events, and they could form the basis of a routine social listening program for health preparedness and response planning.

The COVID-19 pandemic is the first to unfold in the highly digitalized society of the 21st century and is therefore the first pandemic to benefit from and be threatened by a thriving real-time digital information ecosystem. For this reason, the response to the infodemic required development of a public health social listening taxonomy, a structure that can simplify the chaotic information ecosystem to enable an adaptable monitoring infrastructure that detects signals of fertile ground for misinformation and guides trusted sources of verified information to fill in information voids in a timely manner. A weekly analysis of public online conversations since 23 March 2020 has enabled the quantification of running shifts of public interest in public health-related topics concerning the pandemic and has demonstrated the frequent resumption of information voids relevant for public health interventions and risk communication in an emergency response setting.

BACKGROUND The COVID-19 pandemic has been accompanied by an information epidemic or “infodemic”: too much information including false or misleading information in digital and physical environments during an acute public health event, which leads to confusion, risk-taking and behaviors that can harm health, and lead to mistrust in health authorities and public health response. The analytical method described is part of the WHO work to develop tools for an evidence-based response to the infodemic, enabling prioritization of health response activities. OBJECTIVE The aim of this work was to develop a practical, structured approach to identifying narratives in public online conversations on social media platforms where concerns or confusion exist or where narratives are gaining traction, and to provide actionable data to help WHO prioritize its risk communications efforts where it is most critical in addressing the COVID-19 infodemic. METHODS We developed a taxonomy to filter global COVID-19 public online conversations in social media content in English and French into five themes, with 35 sub themes. The taxonomy and its implementation were validated for retrieval precision and retrieval recall, and reviewed and adapted as the linguistic expression about the pandemic in online conversations changed over time. The aggregated data were analyzed for each sub themes by volume, velocity and the presence of questions, on a weekly basis, to detect signals of information voids where there was potential for confusion or for mis- or dis-information to thrive. A human analyst reviewed the themes for potential information voids and used quantitative data to provide context and insight on narratives, influencers and public reactions. RESULTS A COVID-19 public health social listening taxonomy was developed and applied. A weekly analysis of public online conversations since 23 March 2020 has enabled the quantification of shifts of public interest in public health-related topics concerning the pandemic and has demonstrated the frequent resumption of information voids with verified health information. This approach therefore focuses on infodemic signal detection for actionable intelligence to rapidly inform decision-making for a more effective response, including adapting risk communication. CONCLUSIONS This approach been successfully applied during the COVID-19 pandemic to identify and take action on information voids based on analysis of infodemic signals. More broadly, the results have demonstrated the importance of ongoing monitoring and analysis of public online conversations, as information voids frequently resume and narratives shift over time. The approach is already being piloted in individual countries and WHO regions to generate localized insights and actions, while a pilot of an AI social listening platform is using this taxonomy to aggregate and compare online conversations across 20 countries. Looking beyond the COVID-19 pandemic, the taxonomy and methodology have the potential to be adapted for fast deployment in future public health events.