Evaluating meta-analytic methods to detect selective reporting in the presence of dependent effect sizes.

Rodgers, Melissa A.; Pustejovsky, James E.

doi:10.1037/met0000300

Cited by 269 publications

(264 citation statements)

References 75 publications

Supporting

Mentioning

234

Contrasting

Order By: Relevance

“…Figure 4 presents a forest plot for this analysis. Egger's regression test (incorporating RVE per Rodgers and Pustejovsky, 2020), indicated no evidence of small study bias ( = 0.34, p = 0.244; see Panel B in Figure 2 for a contour-enhanced funnel plot). Because influence diagnostics did not reveal any outliers, a leaveone-out analysis was not conducted.…”

Section: Correlations Between Self-report and Logged Measuresmentioning

confidence: 97%

A Systematic Review and Meta-Analysis of Discrepancies Between Logged and Self- Reported Digital Media Use

Parry¹,

Davidson²,

Sewall³

et al. 2020

Preprint

View full text Add to dashboard Cite

The influence of digital media on personal and social well-being is a question of immense public and academic interest. Scholars in this domain often use retrospective self-report measures of the quantity or duration of media use as a proxy for more objective measures, but the validity of these self-report measures remains unclear. Recent advancements in log-based data collection techniques have produced a growing collection of studies indexing both self-reported media use and device-logged measurements. Herein, we report a meta-analysis of this body of research. Based on 104 effect sizes, we found that self-reported media use was only moderately correlated with device-logged measurements, and that these self-report measures were rarely an accurate reflection of logged media use. These results demonstrate that self-reported measures of the quantity or duration of media use are not a valid index of the amount of time people actually spend using media. These findings have serious implications for the study of media use and well- being, suggesting that cautiousness is warranted in drawing conclusions regarding media effects from studies relying solely on self-reported measures of media use.

show abstract

Section: Correlations Between Self-report and Logged Measuresmentioning

confidence: 97%

A Systematic Review and Meta-Analysis of Discrepancies Between Logged and Self- Reported Digital Media Use

Parry¹,

Davidson²,

Sewall³

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…The previous methods assume independent effect sizes in their original formulation. A way to account for dependence is to combine the all the effect sizes coming from the same sample generating an average estimate for each study, and conduct the classic methods on these aggregated estimates (Rodgers & Pustejovsky, 2020). In addition, some recent approaches directly handle the issue of dependence.…”

Section: Publication Biasmentioning

confidence: 99%

“…In addition, some recent approaches directly handle the issue of dependence. For instance, the logic of PET-PEESE and other regression-based methods can be extended to multilevel models and RVE (Fernández-Castilla et al, 2019;Friese et al, 2017;Rodgers & Pustejovsky, 2020). Mathur and VanderWeele (2020) also proposed a sensitivity analysis that can be fitted with RVE.…”

Section: Publication Biasmentioning

confidence: 99%

Please don’t stop the music: A meta-analysis of the cognitive and academic benefits of instrumental musical training in childhood and adolescence

Román-Caballero¹,

Vadillo²,

Trainor³

et al. 2021

Preprint

View full text Add to dashboard Cite

An extensive literature has investigated the impact of musical training on cognition and academic achievement in children and adolescents. However, most of the studies have relied on cross-sectional designs, which makes it impossible to elucidate whether the observed differences are a consequence of the engagement in musical activities. Previous meta-analyses with longitudinal studies have also found inconsistent results, possibly due to their reliance on vague definitions of musical training. In addition, more evidence has appeared in recent years. The current meta-analysis investigates the impact of early programs that involve learning to play musical instruments on cognitive and academic skills, as previous meta-analyses have not focused on this form of musical training. Following a systematic search, 34 independent samples of children and adolescents were included. All the studies had pre-post designs and, at least, one control group. Overall, we found a small but significant benefit (g = 0.24) with short-term programs, regardless of whether they were randomized or not. In addition, a small advantage at baseline was observed in studies with self-selection (g = 0.28), indicating that participants who had the opportunity to select the activity consistently showed a slightly superior performance prior to the beginning of the intervention. Our findings support a nature and nurture approach to the relationship between instrumental training and cognitive performance. Nevertheless, evidence from well-conducted studies is still scarce and more studies are necessary to reach firmer conclusions.

show abstract

“…Egger's regression tests for dependent effect sizes and funnel plots were used for publication bias detection. While no bias was detected, it should be noted that such tests do not have strong power, especially when the number of articles is small, as in case of our meta-analysis [34].…”

Section: Publication Bias and Sensitivity Analysesmentioning

confidence: 73%

“…We contacted the authors of unpublished studies (with up to three email attempts) and discovered that most of them had indeed been published. We tested for publication bias using modification of Egger's regression test for dependent effect sizes [34].…”

Section: Risk Of Biasmentioning

confidence: 99%

Peer influence in adolescent drinking behavior: A meta-analysis of stochastic actor-based modeling studies

Ivaniushina

Titkova

2021

PLoS ONE

View full text Add to dashboard Cite

Objectives To measure the effects of peer influence and peer selection on drinking behavior in adolescence through a rigorous statistical approach designed to unravel these interrelated processes. Methods We conducted systematic searches of electronic databases, thesis collections and conference proceedings to identify studies that used longitudinal network design and stochastic actor-oriented modeling to analyze drinking behavior in adolescents. Parameter estimates collected from individual studies were analyzed using multilevel random-effects models. Results We identified 26 articles eligible for meta-analysis. Meta-analyses for different specifications of the peer influence effect were conducted separately. The peer influence effect was positive for every specification: for average similarity (avSim) mean log odds ratio was 1.27 with 95% confidence interval [0.04; 2.49]; for total similarity (totSim) 0.46 (95% CI = [0.44; 0.48]), and for average alter (avAlt) 0.70 (95% CI = [-0.01; 1.41]). The peer selection effect (simX) was also positive: 0.46 (95% CI = [0.28; 0.63]). Conversion log odds ratio values to Cohen’s d gives estimates from 0.25 to 0.70, which is considered as medium to large effect. Conclusions Advances in methodology for social network analysis have made it possible to accurately estimate peer influence effects free from peer selection effects. More research is necessary to clarify the roles of age, gender, and individual susceptibility on the changing behavior of adolescents under the influence of their peers. Understanding the effects of peer influence should inform practitioners and policy makers to design and deliver more effective prevention programs.

show abstract

Evaluating meta-analytic methods to detect selective reporting in the presence of dependent effect sizes.

Cited by 269 publications

References 75 publications

A Systematic Review and Meta-Analysis of Discrepancies Between Logged and Self- Reported Digital Media Use

A Systematic Review and Meta-Analysis of Discrepancies Between Logged and Self- Reported Digital Media Use

Please don’t stop the music: A meta-analysis of the cognitive and academic benefits of instrumental musical training in childhood and adolescence

Peer influence in adolescent drinking behavior: A meta-analysis of stochastic actor-based modeling studies

Contact Info

Product

Resources

About