A comparative study of the impacts of unbalanced sample sizes on the four synthesized methods of meta-analytic structural equation modeling

Abstract

BackgroundIn the first stage of meta-analytic structural equation modeling (MASEM), researchers synthesized studies using univariate meta-analysis (UM) and multivariate meta-analysis (MM) approaches. The MM approaches are known to be of better performance than the UM approaches in the meta-analysis with equal sized studies. However in real situations, where the studies might be of different sizes, the empirical performance of these approaches is yet to be studied in the first and second stages of MASEM. The present study aimed to evaluate the performance of the UM and MM methods, having unequal sample sizes in different primary studies. Testing the homogeneity of correlation matrices and the empirical power, estimating the pooled correlation matrix and also, estimating parameters of a path model were investigated using these approaches by simulation.ResultsThe results of the first stage showed that Type I error rate was well under control at 0.05 level when the average sample sizes were 200 or more, irrespective of the types of the methods or the sample sizes used. Moreover, the relative percentage biases of the pooled correlation matrices were also lower than 2.5% for all methods. There was a dramatic decrease in the empirical power for all synthesis methods when the inequality of the sample sizes was increased. In fitting the path model at the second stage, MM methods provided better estimation of the parameters.ConclusionsThis study showed the different performance of the four methods in the statistical power, especially when the sample sizes of primary studies were highly unequal. Moreover, in fitting the path model, the MM approaches provided better estimation of the parameters.Electronic supplementary materialThe online version of this article (doi:10.1186/s13104-017-2768-5) contains supplementary material, which is available to authorized users.

Attachment at Work: A Meta-Analysis

“…First, the meta-analytic structural equation modeling technique we utilized here (Viswesvaran & Ones, 1995) has some notable limitations. Specifically, this method suffers from issues associated with unbalanced sample sizes of the coefficients included in the correlation matrix (Alamolhoda et al, 2017). Future research could replicate our findings using other methods that can alleviate this heterogeneity problem, such as two-stage structural equation modeling (Cheung & Chan, 2009) or full information meta-analytical structural equation modeling (Yu et al, 2018).…”

Identifying novel high-impact rare disease-causing mutations, genes and pathways in exomes of Ashkenazi Jewish inflammatory bowel disease patients

“…5 of these genes were replicated from the SKAT-O CD specific analyses: ITPR3, NOD2, CST5, MUC21 and PRAM1 . No significant genes were identified in UC-specific meta analyses, as the unbalanced sample size between UC cases to controls dramatically decreased the power of meta-analysis 41 . CST5 has been shown to display significant differential expression in IBD patients’ plasma levels, with higher expression in CD as compared to UC plasma 42 .…”

Functional Annotation of Human Cognitive States using Deep Graph Convolution

“…Researchers have attempted to tackle the issue of domain-general brain decoding by using meta-analytic 489 approaches based on thousands of reported brain coordinates (Bartley et al , 2018) and a set of statistical contrast maps (Rubin et al , no date) from a series of published studies. But meta-analyses bring other types of limitations, such as unbalanced samples across different cognitive domains (Alamolhoda, Ayatollahi and Bagheri, 2017) , publication bias towards positive effects (Dubben and Beck-Bornholdt, 2005) , as well as over-estimated effect sizes from small studies (Lin, 2018) . These factors may bias the decoding analysis by falsely inferring the mental states given limited available studies (see discussion in (Lieberman and Eisenberger, 2015;Lieberman et al , 2016;Wager et al , 2016) ).…”