Author Correction: Comparing meta-analyses and preregistered multiple-laboratory replication projects

Kvarven, Amanda; Strømland, Eirik; Johannesson, Magnus

doi:10.1038/s41562-020-0864-3

Cited by 10 publications

(24 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“… We will replicate meta-analytic findings that autistic people have: higher pitch mean and variability; more frequent and longer pauses; no differences in speech rate and syllable length, compared to neurotypical participants. Effect sizes will be half to a third smaller than previous meta-analytic findings due to hard to correct publication bias issues (Kvarven et al, 2020; Lewis et al, 2020). See Table 2 for effect sizes of meta-analytic findings. At least some of the measures of voice quality will be different in autistic people compared to neurotypicals, with effect sizes comparable to prosodic measures. We expect the acoustic profile of autistic voice to be affected by individual differences (vs. a unique profile of autistic voice).…”

Section: Introductionmentioning

confidence: 75%

Section: Introductionmentioning

confidence: 75%

“…More generally our study provides a concrete case of cumulative yet self-critical scientific approach, which might extend beyond the specific phenomenon investigated. The need to systematically rely on previous literature (e.g., via systematic reviews and meta-analyses) has been argued to stand in contrast with the potential unreliability of the current literature due to questionable research practices, publication bias and other issues (Bavel et al, 2016; Benjamin et al, 2018; Fabrigar et al, 2020; Kenny & Judd, 2019; Kvarven et al, 2020; Loken & Gelman, 2017; Maxwell et al, 2015; Oberauer & Lewandowsky, 2019; Yarkoni, 2020). In this study, we suggest a way to account for both sides.…”

Section: Discussionmentioning

confidence: 99%

“…a. Effect sizes will be half to a third smaller than previous meta-analytic findings due to hard to correct publication bias issues (Kvarven et al, 2020;Lewis et al, 2020). See Table 2 for effect sizes of meta-analytic findings.…”

Section: Hypothesesmentioning

confidence: 94%

See 3 more Smart Citations

Towards a cumulative science of vocal markers of autism: a cross-linguistic meta-analysis-based investigation of acoustic markers in American and Danish autistic children

Fusaroli

Grossman

Bilenberg

et al. 2021

Preprint

View full text Add to dashboard Cite

Acoustic atypicalities in speech production are widely documented in Autism Spectrum Disorder (ASD) and argued to be both a potential factor in atypical social development and potential markers of clinical features. A recent meta-analysis highlighted shortcomings in the field, in particular small sample sizes and study heterogeneity (Fusaroli, Lambrechts, Bang, Bowler, & Gaigg, 2017). We showcase a cumulative yet self-correcting approach to prosody in ASD to overcome these issues. We analyze a cross-linguistic corpus of multiple speech productions in 77 autistic children and adolescents and 72 TD ones (>1000 recordings in Danish and US English). We replicate findings of a minimal cross-linguistically reliable distinctive acoustic profile for ASD (higher pitch and longer pauses) with moderate effect sizes. We identified novel general reliable differences between the two groups for normalized amplitude quotient, maxima dispersion quotient and creakiness. However, all these relations are small, and there is likely no one general extensive acoustic profile characterizing all autistic individuals. We identified reliable and consistent relations of acoustic features with individual differences (age, gender), and clinical feature: speech rate and ADOS sub-scores (Communication, Social, Stereotyped). Besides cumulatively building our understanding of acoustic atypicalities in ASD, the study concretely shows how to use systematic reviews and meta-analyses to guide follow-up studies, both in their design and their statistical inferences. We indicate future directions: larger and more diverse cross-linguistic datasets, use of previous findings as statistical priors, understanding of covariance between acoustic measures, reliance on machine learning procedures, and open science.

show abstract

Section: Introductionmentioning

confidence: 75%

Section: Introductionmentioning

confidence: 75%

Section: Discussionmentioning

confidence: 99%

Section: Hypothesesmentioning

confidence: 94%

See 2 more Smart Citations

Towards a cumulative science of vocal markers of autism: a cross-linguistic meta-analysis-based investigation of acoustic markers in American and Danish autistic children

Fusaroli

Grossman

Bilenberg

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“… Note : See Kvarven, Strømland, and Johannesson's 4,51 https://static-content.springer.com/esm/art:10.1038/s41562-019-0787-z/MediaObjects/41562_2019_787_MOESM1_ESM.pdf for data, codes and full references. ES‐rep is the replicated effect size and its z ‐value for testing a zero effect is in parenthesis.…”

Section: Are Meta‐analyses Credible?ixmentioning

confidence: 99%

Retrospective median power, false positive meta‐analysis and large‐scale replication

Stanley

Doucouliagos

Ioannidis

2021

Research Synthesis Methods

View full text Add to dashboard Cite

Recent, high‐profile, large‐scale, preregistered failures to replicate uncover that many highly‐regarded experiments are “false positives”; that is, statistically significant results of underlying null effects. Large surveys of research reveal that statistical power is often low and inadequate. When the research record includes selective reporting, publication bias and/or questionable research practices, conventional meta‐analyses are also likely to be falsely positive. At the core of research credibility lies the relation of statistical power to the rate of false positives. This study finds that high (>50%–60%) median retrospective power (MRP) is associated with credible meta‐analysis and large‐scale, preregistered, multi‐lab “successful” replications; that is, with replications that corroborate the effect in question. When median retrospective power is low (<50%), positive meta‐analysis findings should be interpreted with great caution or discounted altogether.

show abstract

Toward a cumulative science of vocal markers of autism: A cross‐linguistic meta‐analysis‐based investigation of acoustic markers in American and Danish autistic children

et al. 2021

View full text Add to dashboard Cite

Acoustic atypicalities in speech production are argued to be potential markers of clinical features in autism spectrum disorder (ASD). A recent meta-analysis highlighted shortcomings in the field, in particular small sample sizes and study heterogeneity. We showcase a cumulative (i.e., explicitly building on previous studies both conceptually and statistically) yet self-correcting (i.e., critically assessing the impact of cumulative statistical techniques) approach to prosody in ASD to overcome these issues. We relied on the recommendations contained in the meta-analysis to build and analyze a cross-linguistic corpus of multiple speech productions in 77 autistic and 72 neurotypical children and adolescents (>1000 recordings in Danish and US English). We used meta-analytically informed and skeptical priors, with informed priors leading to more generalizable inference. We replicated findings of a minimal cross-linguistically reliable distinctive acoustic profile for ASD (higher pitch and longer pauses) with moderate effect sizes. We identified novel reliable differences between the two groups for normalized amplitude quotient, maxima dispersion quotient, and creakiness. However, the differences were small, and there is likely no one acoustic profile characterizing all autistic individuals. We identified reliable relations of acoustic features with individual differences (age, gender), and clinical features (speech rate and ADOS subscores). Besides cumulatively building our understanding of acoustic atypicalities in ASD, the study shows how to use systematic reviews and meta-analyses to guide the design and analysis of follow-up studies. We indicate future directions: larger and more diverse cross-linguistic datasets, focus on heterogeneity, selfcritical cumulative approaches, and open science. Lay SummaryAutistic individuals are reported to speak in distinctive ways. Distinctive vocal production can affect social interactions and social development and could represent a noninvasive way to support the assessment of autism spectrum disorder (ASD). We systematically checked whether acoustic atypicalities highlighted in previous articles could be actually found across multiple recordings and two languages. We find a minimal acoustic profile of ASD: higher pitch, longer pauses, increased hoarseness and creakiness of the voice. However, there is much individual variability (by age, sex, language, and clinical characteristics). This suggests that the search for one common "autistic voice" might be naive and more finegrained approaches are needed.

show abstract

Author Correction: Comparing meta-analyses and preregistered multiple-laboratory replication projects

Abstract: In the original version of this article, there was one code error and some data errors that required updating of Figs. 2-6, Table 1 and the Supplementary Information file. The core conclusions are unaffected, but some numbers and statements have been updated.

Cited by 10 publications

References 0 publications

Towards a cumulative science of vocal markers of autism: a cross-linguistic meta-analysis-based investigation of acoustic markers in American and Danish autistic children

Towards a cumulative science of vocal markers of autism: a cross-linguistic meta-analysis-based investigation of acoustic markers in American and Danish autistic children

Retrospective median power, false positive meta‐analysis and large‐scale replication

Toward a cumulative science of vocal markers of autism: A cross‐linguistic meta‐analysis‐based investigation of acoustic markers in American and Danish autistic children

Contact Info

Product

Resources

About