Community-Augmented Meta-Analyses (CAMAs) in Psychology

Burgard, Tanja; Bošnjak, Michael; Studtrucker, Robert

doi:10.1027/2151-2604/a000431

Cited by 17 publications

(29 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…A CAMA is supposed to facilitate and foster the accumulation of evidence by providing a user‐friendly infrastructure to the research community. Existing systems in psychology have been reviewed and presented in a previous article 17 . In essence, four systems were compared: The crowdsourced platform Curate Science 18 allows the permanent curation of findings by the research community and enables a systematic evaluation of empirical research, primarily in cognitive and social psychology.…”

Section: Community‐augmented Meta‐analysismentioning

confidence: 99%

PsychOpen CAMA: Publication of community‐augmented meta‐analyses in psychology

Burgard

Bošnjak

Studtrucker

2021

Research Synthesis Methods

Self Cite

View full text Add to dashboard Cite

To enable optimal decision‐making based on the best evidence available, open syntheses are called for. To make data accessible and comprehensible even for decision‐makers without proficient knowledge in meta‐analysis, a graphical user interface (GUI) provides flexible data visualizations including interpretation aids. Moreover, due to a growing number of research findings, efficient and easy updating of meta‐analyses is crucial to prevent waste in research. One label for a concept to meet these needs is community‐augmented meta‐analysis (CAMA). The research community at the one hand feeds the data repository of a CAMA with new data and on the other hand benefits from easy access to data and meta‐analyses on a GUI. PsychOpen CAMA has been released recently to serve the psychological research community as a whole by covering a broad scope of potential research domains. PsychOpen CAMA relies on a web application with an OpenCPU server for the R calculations. To achieve interoperability of different datasets with the analysis functions used in PsychOpen CAMA, a template for meta‐analytic data and machine‐readable metadata are used. In the future, the automation of workflows, flexibility of analysis options, and the scope of the platform will be further developed by making use of synergies with other resources and tools at ZPID. The article provides an overview on the rationale for the necessity of open syntheses and the CAMA approach, as well as a presentation of the architecture, user interface, functionalities and future challenges of PsychOpen CAMA.

show abstract

Section: Community‐augmented Meta‐analysismentioning

confidence: 99%

PsychOpen CAMA: Publication of community‐augmented meta‐analyses in psychology

Burgard

Bošnjak

Studtrucker

2021

Research Synthesis Methods

Self Cite

View full text Add to dashboard Cite

show abstract

“…Against the background of an ongoing replication crisis in psychology ( Open Science Collaboration, 2015 ; Świątkowski and Dompnier, 2017 ; Klein et al, 2018 ), we have argued that PLS may be a powerful tool for rebuilding public trust in psychological science, and first experimental findings ( Kerwer et al, 2021 ) seem to support this notion (see also Carvalho et al, 2019 for non-experimental findings that point in the same direction). Because meta-analyses possess a higher quality of evidence and are instrumental for summarizing findings and guiding practitioners ( Borenstein et al, 2009 ; Bastian et al, 2010 ), we further argue that the lay-friendly communication of psychological meta-analytical findings is of particular importance – even though legitimate criticisms on the validity of their findings may exist (e.g., Sharpe and Poets, 2020 ; Burgard et al, 2021 ; Patall, 2021 ).…”

Section: Introductionmentioning

confidence: 94%

“…For this purpose, Study 2 also had two study arms. In study arm A, two PLS with comparably low complexity were presented: One PLS reported only one correlation coefficient as an effect size (based on a meta-analysis by Schwalm et al, 2021 , PLS_LC1), and one PLS reported only one mean difference as an effect size (based on a meta-analysis by Bucher et al, 2020 ; in the ZPID’s PsychOpen CAMA system, see Burgard et al, 2021 , PLS_LC2). Study arm B included two entirely different PLS with comparably high complexity: One PLS reported meta-moderator analyses (based on a meta-analysis by Bergmann and Cristia, 2016 , PLS_HC1), and one PLS reported nine correlation coefficients (based on a meta-analysis by Yule et al, 2019 , PLS_HC2).…”

Section: Studymentioning

confidence: 99%

How to Put It Plainly? Findings From Two Randomized Controlled Studies on Writing Plain Language Summaries for Psychological Meta-Analyses

et al. 2021

View full text Add to dashboard Cite

Plain language summaries (PLS) aim to communicate research findings to laypersons in an easily understandable manner. Despite the societal relevance of making psychological research findings available to the public, our empirical knowledge on how to write PLS of psychology studies is still scarce. In this article, we present two experimental studies investigating six characteristics of PLS for psychological meta-analyses. We specifically focused on approaches for (1) handling technical terms, (2) communicating the quality of evidence by explaining the methodological approach of meta-analyses, (3) explaining how synthesized studies operationalized their research questions, (4) handling statistical terms, (5) structuring PLS, and (6) explaining complex meta-analytic designs. To develop empirically validated guidelines on writing PLS, two randomized controlled studies including large samples stratified for education status, age, and gender (NStudy1=2,288 and NStudy2=2,211) were conducted. Eight PLS of meta-analyses from different areas of psychology were investigated as study materials. Main outcome variables were user experience (i.e., perceived accessibility, perceived understanding, and perceived empowerment) and knowledge acquisition, as well as understanding and knowledge of the quality of evidence. Overall, our hypotheses were partially confirmed, with our results underlining, among other things, the importance of explaining or replacing content-related technical terms (i.e., theoretical concepts) and indicating the detrimental effects of providing too many details on statistical concepts on user experience. Drawing on these and further findings, we derive five empirically well-founded rules on the lay-friendly communication of meta-analytic research findings in psychology. Implications for PLS authors and future research on PLS are discussed.

show abstract

“…There exist several open science repositories, such as the Open Science Foundation (OSF; for a tutorial, see Soderberg 2018), to preregister and make documents publicly available. Furthermore, several initiatives in the social sciences have been established to develop dynamic metaanalyses, such as metaBUS (Bosco et al 2015(Bosco et al , 2017, MetaLab (Bergmann et al 2018), or PsychOpen CAMA (Burgard et al 2021).…”

Section: Open-science Practicesmentioning

confidence: 99%