2021
DOI: 10.1162/opmi_a_00048
|View full text |Cite
|
Sign up to set email alerts
|

Toward Cumulative Cognitive Science: A Comparison of Meta-Analysis, Mega-Analysis, and Hybrid Approaches

Abstract: There is increasing interest in cumulative approaches to science, in which instead of analyzing the results of individual papers separately, we integrate information qualitatively or quantitatively. One such approach is meta-analysis, which has over 50 years of literature supporting its usefulness, and is becoming more common in cognitive science. However, changes in technical possibilities by the widespread use of Python and R make it easier to fit more complex models, and even simulate missing data. Here we … Show more

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...
2
1
1

Citation Types

0
10
0

Year Published

2022
2022
2025
2025

Publication Types

Select...
6
1

Relationship

0
7

Authors

Journals

citations
Cited by 14 publications
(10 citation statements)
references
References 29 publications
0
10
0
Order By: Relevance
“…Their deeper understanding is a long-term task for the discipline that can be greatly facilitated with the methods of cumulative open science. As researchers generate more data in in situ spatial cognition studies (for various research questions), assumptions about the existence of systematic biases could be formally tested in mega-analyses (Koile and Cristia 2021 )—an alternative to meta-analysis that uses unaggregated raw data from past studies. However, as for now too little datasets in the field are open.…”
Section: Discussionmentioning
confidence: 99%
“…Their deeper understanding is a long-term task for the discipline that can be greatly facilitated with the methods of cumulative open science. As researchers generate more data in in situ spatial cognition studies (for various research questions), assumptions about the existence of systematic biases could be formally tested in mega-analyses (Koile and Cristia 2021 )—an alternative to meta-analysis that uses unaggregated raw data from past studies. However, as for now too little datasets in the field are open.…”
Section: Discussionmentioning
confidence: 99%
“…Whereas developmental psychologists were constrained to collecting data in laboratory settings, our toolkit has expanded to include online testing (Chuey et al, 2021;Scott & Schulz, 2017), large-scale replication (Frank et al, 2017), and meta-and mega-analysis of existing data (Koile & Cristia, 2021;Tsuji et al, 2017). Developmental psychology has also partnered with the field of computational cognitive science to formalize theories about knowledge and learning, and design tests of those theories (Lake et al, 2017;Shu et al, 2021;Smith et al, 2019;Tenenbaum et al, 2011).…”
Section: Discussionmentioning
confidence: 99%
“…In this case, the researcher will need additional modeling steps, for instance using the body of previous literature in a rawer form. Tools at this stage include Individual Participant Data (IPD) meta-analyses (Riley et al, 2020;Verhage et al, 2020); mega-analyses (Sung et al, 2014); and hybrid meta-and mega-analyses or pseudo-IPD meta-analyses (Koile and Cristia, 2021;Papadimitropoulou et al, 2019). All points discussed here apply to these different formats of quantitatively aggregating evidence, but for simplicity we limit our considerations to summarizing group-level data.…”
Section: How Data Are Currently Used In Theory Evaluationmentioning
confidence: 99%