True and error analysis instead of test of correlated proportions: Can we save lexicographic semiorder models with error theory?

Abstract

This article criticizes conclusions drawn from the standard test of correlated proportions when the dependent measure contains error. It presents a tutorial on a new method of analysis based on the true and error (TE) theory. This method allows the investigator to separate measurement of error from substantive conclusions about the effects of the independent variable, but it requires replicated measures of the dependent variable. The method is illustrated with hypothetical examples and with empirical data from tests of lexicographic semiorder (LS) models proposed as descriptive theories of risky decision making. LS models imply a property known as interactive independence. Data from two previous studies are reanalyzed to test interactive independence. The new analyses yielded clear answers: interactive independence can be rejected; therefore, LSs can be rejected as descriptive, even when the most flexible error model is allowed. The new methods of analysis can be applied to situations in which the test of correlated proportions would be applied, where it is possible to obtain repeated measures.

Dominance and transitivity

“…A family of models known as 'TE' models have been developed in a series of papers to address the problem of distinguishing true violations from those that might be due to error (Birnbaum, 2004(Birnbaum, , 2008(Birnbaum, , 2010(Birnbaum, , 2013(Birnbaum, , 2023a(Birnbaum, , 2023bBirnbaum et al, 2016;Bahra, 2012a, 2012b;Birnbaum and Gutierrez, 2007;Birnbaum and Quispe-Torreblanca, 2018;Birnbaum and Schmidt, 2008;Birnbaum and Wan, 2020). Research using replicated designs analyzed by TE models concluded that most people satisfy transitivity of preference, but a minority of participants can be found who exhibit systematic deviations from transitivity in specially constructed designs (Birnbaum, 2023a;Birnbaum and Bahra, 2012b;Birnbaum and Diecidue, 2015;Birnbaum and Gutierrez, 2007).…”