Accumulating evidence using crowdsourcing and machine learning: A living bibliography about existential risk and global catastrophic risk

Innovation and forward‐thinking are needed to improve traditional synthesis methods: A response to Pescott and Stewart

“…reliability), as different components of the overall validity of a study can be considered. As discussed earlier, the alternative weighting method of 'recalibration' has been proposed in the medical sciences (Kneale et al, 2019) to adjust studies' weights in meta-analyses by their relevance to a decision-maker's question and context of interest (as trialled in dynamic meta-analyses on the Metadataset platform for interventions on invasive species and agricultural management; Shackelford et al, 2020). The issue of external validity and relevance is a crucial issue for decision-makers and the usefulness of evidence syntheses, particularly in disciplines like ecology and conservation science, where variation between studies based on their local context (e.g.…”

“…Given that the scientific literature in most disciplines is growing rapidly (Bornmann & Mutz, 2015; Larsen & von Ins, 2010) and that publication delays already hamper evidence‐based decision‐making in crisis disciplines, such as conservation (Christie et al, 2021), evidence synthesis needs to be as time‐ and cost‐efficient as possible (Grainger et al, 2020; Nakagawa et al, 2020). Generating large‐scale, easily updateable, ‘living’ evidence databases containing results and metadata of scientific studies across subjects is therefore central to future‐proofing evidence synthesis (Elliott et al, 2021; Shackelford et al, 2020). It is these forward‐thinking approaches to synthesis that can be facilitated by alternative methods of weighting, which can use the metadata on studies to automatically and rapidly critically appraise studies and weight meta‐analyses, without the need for cumbersome, inefficient and ultimately subjective manual risk‐of‐bias assessments that Pescott and Stewart promote.…”

“…Traditional methods of synthesis advocated for by Pescott and Stewart typically fail to account for or consider the relevance of different studies to decision‐makers in any detail, and certainly do not directly integrate these meta‐analytic weightings of studies. As different studies will have different levels of relevance to different decision‐makers, these alternative weighting methods also support the movement towards more interactive, living meta‐analytic platforms for evidence synthesis (rather than static publications of meta‐analyses) that can be dynamically adjusted to give bespoke evidence‐based recommendations (Kneale et al, 2019; Shackelford et al, 2020). Developing alternative weighting methods, rather than avoiding them, is therefore central to increasing the usefulness and relevance of evidence syntheses to decision‐makers.…”

Societal collapse: A literature review

“…These summaries charted different models of collapse based on past societies (Cumming & Peterson, 2017;Faulseit, 2016), systematically scrutinized global warming effects on public health (Butler, 2018), reviewed research on collapses in specific geographical areas (e.g., Dunning et al, 2015;Kaniewski et al, 2015;Sheridan, 2017;Jones et al, 2019), attempted to integrate collapse theories (Rubiños & Anderies, 2020), addressed transformative adaptations of past societies faced with disasters (Ajulo et al, 2020), presented practices and strategies for dealing with collapse from the perspective of systems science (Bardi, 2020), and attempted to identify possible causes of collapse (Shackelford et al, 2020). Other reviews have also touched upon topics such as transformative adaptation after disasters (Ajulo et al, 2020;Feola, 2015).…”

Automated synthesis of biodiversity knowledge requires better tools and standardised research output

“…'Big data' approaches, and associated computational tools, provide a means to wrangle the extensive ecological literature into usable information (Westgate et al 2018). Much of the recent development in synthesis methods has been in expediting and automating the searching for (Grames et al 2019), and screening of (Wallace et al 2012, Shackelford et al 2020, Cornford et al 2021, papers to address research questions. Within the medical literature, some approaches have even managed to automate the entire systematic review procedure (Marshall and Wallace 2019, Gates et al 2020, Marshall et al 2020, Yang et al 2020, Brassey et al 2021).…”