Issues, problems and potential solutions when simulating continuous, non-normal data in the social sciences

Abstract

Computer simulations have become one of the most prominent tools for methodologists in the social sciences to evaluate the properties of their statistical techniques and to offer best practice recommendations. Amongst the many uses of computer simulations, evaluating the robustness of methods to their assumptions, particularly univariate or multivariate normality, is crucial to ensure the appropriateness of data analysis. In order to accomplish this, quantitative researchers need to be able to generate data where they have a degree of control over its non-normal properties. Even though great advances have been achieved in statistical theory and computational power, the task of simulating multivariate, non-normal data is not straightforward. There are inherent conceptual and mathematical complexities implied by the phrase ``non-normality'' which are not always reflected in the simulations studies conduced by social scientists. The present article attempts to offer a summary of some of the issues concerning the simulation of multivariate, non-normal data in the social sciences. An overview of common algorithms is presented as well as some of the characteristics and idiosyncrasies that implied in them which may exert undue influence in the results of simulation studies. A call is made to encourage the meta-scientific study of computer simulations in the social sciences in order to understand how simulation designs frame the teaching, usage and practice of statistical techniques within the social sciences.

Theoretical considerations when simulating data from the g‐and‐h family of distributions

“…The small-scale simulation study shown in Section 3 highlights an important aspect of simulation research as it is conducted in the social sciences: there is a differential effect that the method of simulation has on simulation results. Although there is very little research on this, Astivia andZumbo (2015, 2018) and Falk (2018) have shown that the data-generating method can be considered as a simulation condition in itself. Their results support the fact that, even for the same marginal non-normality, different ways to generate the data can result in different conclusions for computer studies.…”

“…An important feature of that allows it to become a true multivariate generalization is that univariate normal distributions are closed under convolutions, that is, linear combinations of normally distributed random variables result in another normally distributed random variable. If the random variables being correlated do not posses this property, the central limit theorem can take over and weaken the non‐normality aspects that researchers are interested in using as simulation conditions (Astivia, 2020).…”

“…In Kowalchuk and Headrick (2010), one begins by sampling from a multivariate normal distribution where each one-dimensional marginal is then transformed using equations (1), ( 2) or (3) depending on the researcher's needs. Navely implementing this method would change the covariance/correlation structure specified in the initial multivariate normal distribution, biasing the final correlations downwards (see Astivia, 2020). To bypass this issue, Kowalchuk and Headrick (2010) offer a correction to ensure that the final correlation structure is preserved as well as the nature of the lower-dimensional distributions.…”

Simulating Factor Structures from Continuous and Discrete Distributions using Mixture of Means within a Hierarchical Model

“…Simulation of correlated multivariate normal data, including specification of factor loadings, is simple. Simulation of non-normal correlated data, however, is fraught with issues (Astivia, 2020). The most common method for simulating continuous data is the power method of Fleishman (1978), extended in Vale and Maurelli (1983), Headrick and Sawilowsky (1999), and Headrick (2002).…”

“…The most common method for simulating continuous data is the power method of Fleishman (1978), extended in Vale and Maurelli (1983), Headrick and Sawilowsky (1999), and Headrick (2002). But as Astivia (2020) notes, there are a multiplicity of solutions with no clear guidance to determine which solution is correct. Other proposed methods include using transformations of univariate random variables in Mattson (1997), an iterative algorithm in Ruscio and Kaczetow (2008), vines in Grønneberg and Foldnes (2017), and piecewise linear transforms in Foldnes and Grønneberg (2022), and many others.…”