Psychologists use scales comprised of multiple items to measure underlying constructs. Missing data on such scales often occur at the item level, whereas the model of interest to the researcher is at the composite (scale score) level. Existing analytic approaches cannot easily accommodate item-level missing data when models involve composites. A very common practice in psychology is to average all available items to produce scale scores. This approach, referred to as available-case maximum likelihood (ACML), may produce biased parameter estimates. Another approach researchers use to deal with item-level missing data is scale-level full information maximum likelihood (SL-FIML), which treats the whole scale as missing if any item is missing. SL-FIML is inefficient and it may also exhibit bias. Multiple imputation (MI) produces the correct results using a simulation-based approach. We study a new analytic alternative for item-level missingness, called two-stage maximum likelihood (TSML; Savalei & Rhemtulla, Journal of Educational and Behavioral Statistics, 42(4), 405-431. 2017). The original work showed the method outperforming ACML and SL-FIML in structural equation models with parcels. The current simulation study examined the performance of ACML, SL-FIML, MI, and TSML in the context of univariate regression. We demonstrated performance issues encountered by ACML and SL-FIML when estimating regression coefficients, under both MCAR and MAR conditions. Aside from convergence issues with small sample sizes and high missingness, TSML performed similarly to MI in all conditions, showing negligible bias, high efficiency, and good coverage. This fast analytic approach is therefore recommended whenever it achieves convergence. R code and a Shiny app to perform TSML are provided.

A key prediction of Terror Management Theory is that people affirm their cultural worldview after they are reminded of death. This mortality salience (MS) hypothesis has been widely explored, yet the presence of questionable research practices may impact the replicability of this literature. We assess the evidential value of the MS hypothesis by conducting a pre-registered p-curve analysis of 860 published studies. Our results suggest that there are nonzero effects in this literature and that power is larger for studies conducted with multiple delays between the independent and dependent variables, for studies that test for main effects in comparison to those that test for interactions, and for studies conducted more recently. However, since the estimated average power of MS studies is 26%, direct replications are unlikely to succeed. We recommend researchers consider our evidence when planning their samples, and that they anticipate smaller effects by increasing their sample sizes.

We assessed the evidential value of the large literature (k=826 studies) investigating the mortality salience (MS) hypothesis from terror management theory. We employed a multitool assessment approach and reviewed past efforts to replicate experiments testing the MS hypothesis, and conducted a p-curve, a z-curve, and a random effects meta-analysis including bias corrections of the selection model, PET-PEESE, and WAAP-WLS on the studies. Overall, the different meta-analytic tools pointed to conflicting conclusions, reflecting differences in the methodology and philosophy of these tools. Our synthesis of these findings suggests there are true effects underlying some studies of the MS hypothesis, although the effects are highly heterogeneous, and the majority of studies are underpowered. We recommend future replications to assume a smaller effect size (r = .10 ~ .15) and to follow expert guidance in the experimental protocol. Given the conflicting findings that emerged, we suggest that future attempts to evaluate other literatures would benefit from a multitool assessment approach.

This is the supplementary material for the manuscript entitled "Pay attention to the ignorable missing data mechanisms! An exploration of their impact on the efficiency of regression coefficients".

Extreme groups design (EGD) refers to the use of a screening variable to inform further data collection, such that only participants with the lowest and highest scores are recruited in subsequent stages of the study. It is an effective way to improve the power of a study under a limited budget, but produces biased standardized estimates. We demonstrate that the bias in EGD results from its inherent missing at random mechanism, which can be corrected using modern missing data techniques such as full information maximum likelihood (FIML). Further, we provide a tutorial on computing correlations in EGD data with FIML using R.