How well did experts and laypeople forecast the size of the COVID-19 pandemic?

Recchia, Gabriel; Freeman, Alexandra L. J.; Spiegelhalter, David

doi:10.1371/journal.pone.0250935

Cited by 40 publications

(24 citation statements)

References 42 publications

(39 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Empirical studies show that non-experts are very poor at making predictions about COVID-19, and they are worse than experts—even though experts do not account sufficiently for uncertainty in their estimates and are therefore often also wrong. 21 Both models and empirical data suggest that media can have an impact on the course of the pandemic 22 and it can also affect mental health during its course. 23 While there can be questions and concerns even about the media appearances of the best and most knowledgeable experts, media without involvement of scientific expertise is likely to be far worse.…”

Section: Discussionmentioning

confidence: 99%

Overall and COVID-19-specific citation impact of highly visible COVID-19 media experts: bibliometric analysis

2021

View full text Add to dashboard Cite

ObjectiveTo evaluate whether the COVID-19 experts who appear most frequently in media have high citation impact for their research overall, and for their COVID-19 peer-reviewed publications in particular and to examine the representation of women among such experts.DesignCross-linking of data sets of most highly visible COVID-19 media experts with citation data on the impact of their published work (career-long publication record and COVID-19-specific work).SettingCable news appearance in prime-time programming or overall media appearances.ParticipantsMost highly visible COVID-19 media experts in the USA, Switzerland, Greece and Denmark.InterventionsNone.Outcome measuresCitation data from Scopus along with discipline-specific ranks of overall career-long and COVID-19-specific impact based on a previously validated composite citation indicator.ResultsWe assessed 76 COVID-19 experts who were highly visible in US prime-time cable news, and 50, 12 and 2 highly visible experts in media in Denmark, Greece and Switzerland, respectively. Of those, 23/76, 10/50, 2/12 and 0/2 were among the top 2% of overall citation impact among scientists in the same discipline worldwide. Moreover, 37/76, 15/50, 7/12 and 2/2 had published anything on COVID-19 that was indexed in Scopus as of 30 August 2021. Only 18/76, 6/50, 2/12 and 0/2 of the highly visible COVID-19 media experts were women. 55 scientists in the USA, 5 in Denmark, 64 in Greece and 56 in Switzerland had a higher citation impact for their COVID-19 work than any of the evaluated highly visible media COVID-19 experts in the respective country; 10/55, 2/5, 22/64 and 14/56 of them were women.ConclusionsDespite notable exceptions, there is a worrisome disconnect between COVID-19 claimed media expertise and scholarship. Highly cited women COVID-19 experts are rarely included among highly visible media experts.

show abstract

Section: Discussionmentioning

confidence: 99%

Overall and COVID-19-specific citation impact of highly visible COVID-19 media experts: bibliometric analysis

2021

View full text Add to dashboard Cite

show abstract

“…Previous work has examined such direct human forecasts in various contexts, such as geopolitics (Atanasov et al, 2016;Tetlock et al, 2014), meta-science (Hoogeveen et al, 2020;ReplicationMarkets, 2020), sports (Servan-Schreiber et al, 2004) and epidemiology (Farrow et al, 2017;McAndrew & Reich, 2020;Recchia et al, 2021). Several prediction platforms (CSET Foretell, 2021;Hypermind, 2021;Metaculus, 2020) and prediction markets (PredictIt, 2021) have been created to collate expert and non-expert predictions. However, with the notable exception of Farrow et al (2017), these forecasts were not designed to be evaluated alongside model-based forecasts and usually follow their own (often binary) prediction formats.…”

Section: Introductionmentioning

confidence: 99%

Comparing human and model-based forecasts of COVID-19 in Germany and Poland

Bosse

Abbott

Bracher

et al. 2021

Preprint

View full text Add to dashboard Cite

1AbstractForecasts based on epidemiological modelling have played an important role in shaping public policy throughout the COVID-19 pandemic. This modelling combines knowledge about infectious disease dynamics with the subjective opinion of the researcher who develops and refines the model and often also adjusts model outputs. Developing a forecast model is difficult, resource- and time-consuming. It is therefore worth asking what modelling is able to add beyond the subjective opinion of the researcher alone. To investigate this, we analysed different real-time forecasts of cases of and deaths from COVID-19 in Germany and Poland over a 1-4 week horizon submitted to the German and Polish Forecast Hub. We compared crowd forecasts elicited from researchers and volunteers, against a) forecasts from two semi-mechanistic models based on common epidemiological assumptions and b) the ensemble of all other models submitted to the Forecast Hub. We found crowd forecasts, despite being overconfident, to outperform all other methods across all forecast horizons when forecasting cases (weighted interval score relative to the Hub ensemble 2 weeks ahead: 0.89). Forecasts based on computational models performed comparably better when predicting deaths (rel. WIS 1.26), suggesting that epidemiological modelling and human judgement can complement each other in important ways.

show abstract

“…Computational ensembles during the COVID-19 pandemic have been used to guide the selection and continued evaluation of trial sites for vaccine efficacy studies [36] and have been used to predict the number of individuals fully vaccinated [37] . Human judgment has been applied to predict both epidemiological targets of COVID-19 [38] , [39] and aspects of the various vaccination efforts against COVID-19 [40] , [41] .…”

Section: Introductionmentioning

confidence: 99%