A meta-analytic approach to evaluating the explanatory adequacy of theories

Cristià, Alejandrina; Tsuji, Sho; Bergmann, Christina

doi:10.31219/osf.io/83kg2

Cited by 7 publications

(17 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…All this makes MetaLab a very attractive source of reference data for model evaluation, as it has both up-to-date data and, for most of the capabilities, reports describing the meta-analytic practices to analyze these data for the developmental phenomena of interest, which allows computational modelers that are not experts on infant experimentation to adopt the meta-regression specification decided upon by the meta-analysis authors. In the age of cumulative science, such "community augmented meta-analyses" are likely the best approach to evaluating a theory's explanatory adequacy (i.e., its fit to extant data; Cristia et al, 2021), including because they minimize selection bias, they invite checks for scope (i.e., whether the whole space of design and stimuli is appropriately represented), and they use statistical techniques to actually account for potential effect variation. We will be drawing from MetaLab for our Experiment #2 (see Section 4).…”

Section: Human Reference From Meta-analysismentioning

confidence: 99%

“…4 for an illustration). Null effects are ambiguous: Although theoretically they can still act as a meaningful target for a model (i.e., the model should also have null effect for this capability), they may also signal noisy infant data (see Section 3.4), which is an issue particularly for moderators (for which power is always lower than for main effects; see also Lewis et al, 2016;and Steps 9 and 0 in Cristia et al, 2021). In the absence of better knowledge, we will treat null effects as null targets for models as well, especially if the null finding is supported by substantial statistical power.…”

Section: Human Reference From Meta-analysismentioning

confidence: 99%

“…Instead, lack of generalization could well be due to the fact that psychological phenomena are sensitive to factors researchers do not treat as "random effects" as often as they do participants, including factors like stimuli, procedure, and even experimenter. As we will explain in more detail below, one response to this issue precisely will be to stop relying on individual studies, and turn instead to groups of studies (e.g., via meta-analyses; Cristia et al, 2021).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Introducing meta-analysis in the evaluation of computational models of infant language development

Blandón¹,

Cristià²,

Räsänen³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Computational models of child language development can help us understand the cognitive underpinnings of the language learning process. One advantage of computational modeling is that is has the potential to address multiple aspects of language learning within a single learning architecture. If successful, such integrated models would help to pave the way for a more comprehensive and mechanistic understanding of language development. However, in order to develop more accurate, holistic, and hence impactful models of infant language learning, the research on models also requires model evaluation practices that allow comparison of model behavior to empirical data from infants across a range of language capabilities. Moreover, there is a need for practices that can compare developmental trajectories of infants to those of models as a function of language experience. The present study aims to take the first steps to address these needs. More specifically, we will introduce the concept of comparing models with large-scale cumulative empirical data from infants, as quantified by meta-analyses conducted across a large number of individual behavioral studies. We start by formalizing the connection between measurable model and human behavior, and then present a basic conceptual framework for meta-analytic evaluation of computational models together with basic guidelines intended as a starting point for later work in this direction. We exemplify the meta-analytic model evaluation approach with two modeling experiments on infant-directed speech preference and native/non-native vowel discrimination. We also discuss the advantages, challenges, and potential future directions of meta-analytic evaluation practices.

show abstract

Section: Human Reference From Meta-analysismentioning

confidence: 99%

Section: Human Reference From Meta-analysismentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Introducing meta-analysis in the evaluation of computational models of infant language development

Blandón¹,

Cristià²,

Räsänen³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…These would be important steps towards the cumulating integration of various definitions, measures, and insights from various data sources in joint frameworks. Chartier, Kline, McCarthy, Nuijten, Dunleavy, & Ledgerwood, 2018;Cristia, Tsuji, & Bergmann, 2020;Irani, 2015). The technology and communication infrastructure is ready to be used (e.g., Crüwell et al, 2019).…”

Section: Cumulating Insights By Joining Forces In Multi-lab Collaboramentioning

confidence: 99%

Specific, situated, intra-individual, dialectical, and open: New avenues to integrate and advance the research on passion for work

Moeller¹

2021

Preprint

View full text Add to dashboard Cite

In almost two decades, the research on passion for work has produced a variety of definitions, measures, and models specifying components, predictors, and outcomes the construct. A challenge for the next generation of research studies will be the integration of these diverse theoretical and methodological approaches with each other and with other fields related to the study of motivation and emotion. This article proposes approaches that shall help to integrate and advance the research on passion for work by implementing recent innovations and insights from the research on related motivational and emotional constructs. The following solutions are proposed: (1) specifying the exact facets of passion for work in theoretical models and measures, (2) using psychometric network models to examine the relationships among the facets and the facets’ relationships with relevant predictors and outcomes, (3) formulating and testing process models that distinguish between state- and trait development and include recursive feedback loops, (4) examining the ambivalent motivation and mixed emotions accompanying passion for work with intra-individual methods, (5) working towards cumulative scientific insights by adopting multi-lab collaborations and other open science practices, and (6) distinguishing between the facets that drive affective thriving and those driving perseverance in the face of obstacles.

show abstract

“…Going beyond "classic" meta-analyses, MetaLab implements so-called Community-Augmented Meta-Analyses (CAMAs; Cristia et al, 2021;, which open metaanalyses to the community. This means a meta-analysis can be updated as new studies emerge or null results are dredged from the file drawer.…”

Section: Using Meta-analyses To Inform Later Researchmentioning

confidence: 99%

Ten easy steps to conducting transparent, reproducible meta-analyses for infant researchers

Gasparini¹,

Tsuji²,

Bergmann³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Meta-analyses provide researchers with an overview of the body of evidence in a topic, with quantified estimates of effect sizes and the role of moderators, and weighting studies according to their precision. We provide a guide for conducting a transparent and reproducible meta-analysis in the field of developmental psychology within the framework of the MetaLab platform, in 10 steps: 1) Choose a topic for your meta-analysis, 2) Formulate your research question and specify inclusion criteria, 3) Preregister and carefully document all stages of your meta-analysis, 4) Conduct the literature search, 5) Collect and screen records, 6) Extract data from eligible studies, 7) Read the data into analysis software and compute effect sizes, 8) Create meta-analytic models to assess the strength of the effect and investigate possible moderators, 9) Visualize your data, 10) Write up and promote your meta-analysis. Meta-analyses can inform future studies, through power calculations, by identifying robust methods and exposing research gaps. By adding a new meta-analysis to MetaLab, datasets across multiple topics of developmental psychology can be synthesized, and the dataset can be maintained as a living, community-augmented meta-analysis to which researchers add new data, allowing for a cumulative approach to evidence synthesis.

show abstract

A meta-analytic approach to evaluating the explanatory adequacy of theories

Cited by 7 publications

References 21 publications

Introducing meta-analysis in the evaluation of computational models of infant language development

Introducing meta-analysis in the evaluation of computational models of infant language development

Specific, situated, intra-individual, dialectical, and open: New avenues to integrate and advance the research on passion for work

Ten easy steps to conducting transparent, reproducible meta-analyses for infant researchers

Contact Info

Product

Resources

About