Scientific Disagreements, Fast Science and Higher-Order Evidence

Friedman, Daniel; Šešelja, Dunja

doi:10.1017/psa.2023.83

Philos. sci.

2023

DOI: 10.1017/psa.2023.83

|View full text |Cite

Scientific Disagreements, Fast Science and Higher-Order Evidence

Daniel Friedman¹,

Dunja Šešelja

Abstract: Scientific disagreements are an important catalyst for scientific progress. But what happens when scientists disagree amidst times of crisis, when we need quick yet reliable policy guidance? In this paper we provide a normative account for how scientists facing disagreement in the context of ‘fast science’ should respond, and how policy makers should evaluate such disagreement. Starting from an argumentative, pragma-dialectic account of scientific controversies, we argue for the importance of ‘higher-order evi… Show more

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...

Citation Types

Supporting

Mentioning

Contrasting

Year Published

2024

Publication Types

Select...

Article2

Relationship

Self Cite0

Independent2

Authors

Journals

Cited by 2 publications

References 54 publications

(90 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

Predicting the replicability of social and behavioural science claims in COVID-19 preprints

Marcoci,

Wilkinson,

Vercammen

et al. 2024

Nat Hum Behav

View full text Add to dashboard Cite

Replications are important for assessing the reliability of published findings. However, they are costly, and it is infeasible to replicate everything. Accurate, fast, lower-cost alternatives such as eliciting predictions could accelerate assessment for rapid policy implementation in a crisis and help guide a more efficient allocation of scarce replication resources. We elicited judgements from participants on 100 claims from preprints about an emerging area of research (COVID-19 pandemic) using an interactive structured elicitation protocol, and we conducted 29 new high-powered replications. After interacting with their peers, participant groups with lower task expertise (‘beginners’) updated their estimates and confidence in their judgements significantly more than groups with greater task expertise (‘experienced’). For experienced individuals, the average accuracy was 0.57 (95% CI: [0.53, 0.61]) after interaction, and they correctly classified 61% of claims; beginners’ average accuracy was 0.58 (95% CI: [0.54, 0.62]), correctly classifying 69% of claims. The difference in accuracy between groups was not statistically significant and their judgements on the full set of claims were correlated (r(98) = 0.48, P < 0.001). These results suggest that both beginners and more-experienced participants using a structured process have some ability to make better-than-chance predictions about the reliability of ‘fast science’ under conditions of high uncertainty. However, given the importance of such assessments for making evidence-based critical decisions in a crisis, more research is required to understand who the right experts in forecasting replicability are and how their judgements ought to be elicited.

show abstract

Predicting the replicability of social and behavioural science claims in COVID-19 preprints

Marcoci,

Wilkinson,

Vercammen

et al. 2024

Nat Hum Behav

View full text Add to dashboard Cite

show abstract

Fast Science

Stegenga

2024

The British Journal for the Philosophy of Science

View full text Add to dashboard Cite

scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.

Contact Info

customersupport@researchsolutions.com

10624 S. Eastern Ave., Ste. A-614

Henderson, NV 89052, USA

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Blog Terms and Conditions API Terms Privacy Policy Contact Cookie Preferences Do Not Sell or Share My Personal Information

Made with 💙 for researchers

Part of the Research Solutions Family.

Scientific Disagreements, Fast Science and Higher-Order Evidence

Cited by 2 publications

References 54 publications

Predicting the replicability of social and behavioural science claims in COVID-19 preprints

Predicting the replicability of social and behavioural science claims in COVID-19 preprints

Fast Science

Contact Info

Product

Resources

About