An Introduction to the General Monotone Model with Application to Two Problematic Data Sets

Abstract

We argue that the mismatch between data and analytical methods, along with common practices for dealing with “messy” data, can lead to inaccurate conclusions. Specifically, using previously published data on racial bias and culture of honor, we show that manifest effects, and therefore theoretical conclusions, are highly dependent on how researchers decide to handle extreme scores and nonlinearities when data are analyzed with traditional approaches. Within LS approaches, statistical effects appeared or disappeared on the basis of the inclusion or exclusion of as little as 1.5% (3 of 198) of the data, and highly predictive variables were masked by nonlinearities. We then demonstrate a new statistical modeling technique called the general monotone model (GeMM) and show that it has a number of desirable properties that may make it more appropriate for modeling messy data: It is more robust to extreme scores, less affected by outlier analyses, and more robust to violations of linearity on both the response and predictor variables compared with a variety of well-established statistical algorithms and frequently possesses greater statistical power. We argue that using procedures that make fewer assumptions about the data, such as GeMM, can lessen the need for researchers to use data-editing strategies (e.g., to apply transformations or to engage outlier analyses) on their data to satisfy often unrealistic statistical assumptions, leading to more consistent and accurate conclusions about data than traditional approaches of data analysis.

Order‐constrained linear optimization

“…While the analyses here have been restricted to comparing OCLO/GeMM to OLS, similar results have been obtained comparing GeMM (with and without the OCLO methodology) to various least‐squares alternatives (including the robust, ridge, Bayesian, negative binomial, Poisson, quasi‐Poisson, and ordinal logit models) when modelling other real‐world data sets (Dougherty & Thomas, ; Dougherty et al ., ). As is the case for the above analyses, GeMM/OCLO almost always outperforms these alternative models in terms of ordinal prediction, and in many cases even outpredicts them in terms of the metric properties (for examples, see Dougherty et al ., ). Similar results were obtained by Luan, Schooler, and Gigerenzer () in an analysis of 39 real‐world data sets: compared to a variety of statistical and heuristic algorithms, GeMM was consistently among the best at out‐of‐sample predictive accuracy.…”

“…On the other hand, in our experience, in cases where OLS and other standard statistical algorithms do outperform GeMM/OCLO, the advantage is quite small. This implies that the cost–benefit trade‐off of using GeMM/OCLO for prediction will tend to favour GeMM/OCLO over other metric statistics in the long run (see Dougherty et al ., ).…”

“…A third reason to prefer GeMM and OCLO over traditional statistical procedures is that, since the maximization of rank‐order fit is invariant to monotone transformation of the criterion variable (see Dougherty et al ., ), it obviates any need to transform the criterion and therefore reduces potential researcher degrees of freedom (Simmons et al ., ) or data‐dependent decisions (Gelman & Loken, ). Note that the ordinal objective function (maximizing τ) specified within OCLO reduces to the GeMM model, which itself is a version of the MRC estimator.…”

“…Moreover, because the GeMM relaxes assumptions about functional form and the error distribution, it should be more resistant to deviations from assumptions (e.g., linearity, normality of errors) required for standard parametric procedures of the GLM family. This implies that the GeMM will not require the types of data editing (transformations and outlier deletion) that might be required before using standard techniques (Dougherty et al ., ).…”

“…Cliff, ). That is, there is no constraint on the least‐squares solution that ensures that it will maximize the rank‐order properties of the modelled data, and the minimization of the least squares can even come at the cost of accurately modelling rank‐order properties (see, for example, Dougherty, Thomas, Brown, Chrabaszcz, & Tidwell, ). A good metric fit may yield a relatively poor fit to ordinal properties, and consequently, inferences related to the ordinal properties from these modelled data may be misleading.…”