Incomplete Tests of Conditional Association for the Assessment of Model Assumptions

Ligtvoet, Rudy

doi:10.1007/s11336-022-09841-1

Cited by 3 publications

(10 citation statements)

References 85 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Coincidentally, CA fares well with Spearman's (Spearman, 1904) idea that intelligence tests have positive correlations and together measure a single general intelligence factor, and Guttman's 'first law of intelligence', stating that any two intelligence items have a nonnegative correlation in any population that is "not artificially selected" (Guttman & Levy, 1991), thus suggesting the items should have nonnegative correlations in any subgroup defined by the other items. CA is hard to test fully because it involves many restrictions even for small item sets (De Gooijer & Yuan, 2011;Ligtvoet, 2022;Yuan & Clarke, 2001). Therefore, we will also discuss conditions that are easier to test, such as the condition that the expected item score increases with the rest score, called marginal monotonicity (MM) (Junker, 1993).…”

Section: Testable Conditions Of the Modelsmentioning

confidence: 99%

“…Junker (1993Junker ( , p. 1372, Junker and Sijtsma (2000) showed that MH ⇒ MM. Ligtvoet (2022) showed that CA ⇒ MM, so MM may be viewed as another incomplete test of CA. MM is an important property because it is conceptually like the idea of a monotone IRF, and for the reader who is unfamiliar with these concepts it may be hard to see how one can have MM without MH.…”

Section: Testable Conditions Of the Modelsmentioning

confidence: 99%

“…Therefore, we will also discuss conditions that are easier to test, such as the condition that the expected item score increases with the rest score, called marginal monotonicity (MM) (Junker, 1993 ). These conditions can be viewed as incomplete tests of CA (Ligtvoet, 2022 ). We will now continue this section with the formal definitions.…”

Section: Models and Testable Conditionsmentioning

confidence: 99%

“…Straat et al ( 2016 ) acknowledged this limitation and proposed an incomplete strategy. Ligtvoet ( 2022 ) gives an excellent review of weaker conditions, which he describes as “incomplete tests of conditional association”.…”

Section: Models and Testable Conditionsmentioning

confidence: 99%

“…An important condition discussed by Ligtvoet ( 2022 ) is multivariate total positivity of order 2 (MTP . The ordinary formulation of this condition is given in Appendix A, but for the present purpose it suffices to note that being MTP implies that (Note the omission of h () here).…”

Section: Models and Testable Conditionsmentioning

confidence: 99%

See 4 more Smart Citations

A Test to Distinguish Monotone Homogeneity from Monotone Multifactor Models

Ellis

Sijtsma

2023

Psychometrika

View full text Add to dashboard Cite

The goodness-of-fit of the unidimensional monotone latent variable model can be assessed using the empirical conditions of nonnegative correlations (Mokken in A theory and procedure of scale-analysis, Mouton, The Hague, 1971), manifest monotonicity (Junker in Ann Stat 21:1359–1378, 1993), multivariate total positivity of order 2 (Bartolucci and Forcina in Ann Stat 28:1206–1218, 2000), and nonnegative partial correlations (Ellis in Psychometrika 79:303–316, 2014). We show that multidimensional monotone factor models with independent factors also imply these empirical conditions; therefore, the conditions are insensitive to multidimensionality. Conditional association (Rosenbaum in Psychometrika 49(3):425–435, 1984) can detect multidimensionality, but tests of it (De Gooijer and Yuan in Comput Stat Data Anal 55:34–44, 2011) are usually not feasible for realistic numbers of items. The only existing feasible test procedures that can reveal multidimensionality are Rosenbaum’s (Psychometrika 49(3):425–435, 1984) Case 2 and Case 5, which test the covariance of two items or two subtests conditionally on the unweighted sum of the other items. We improve this procedure by conditioning on a weighted sum of the other items. The weights are estimated in a training sample from a linear regression analysis. Simulations show that the Type I error rate is under control and that, for large samples, the power is higher if one dimension is more important than the other or if there is a third dimension. In small samples and with two equally important dimensions, using the unweighted sum yields greater power.

show abstract

Section: Testable Conditions Of the Modelsmentioning

confidence: 99%

Section: Testable Conditions Of the Modelsmentioning

confidence: 99%

Section: Models and Testable Conditionsmentioning

confidence: 99%

Section: Models and Testable Conditionsmentioning

confidence: 99%

Section: Models and Testable Conditionsmentioning

confidence: 99%

See 3 more Smart Citations

A Test to Distinguish Monotone Homogeneity from Monotone Multifactor Models

Ellis

Sijtsma

2023

Psychometrika

View full text Add to dashboard Cite

show abstract

Recognize the Value of the Sum Score, Psychometrics’ Greatest Accomplishment

Sijtsma,

Ellis,

Borsboom

2024

Psychometrika

View full text Add to dashboard Cite

The sum score on a psychological test is, and should continue to be, a tool central in psychometric practice. This position runs counter to several psychometricians’ belief that the sum score represents a pre-scientific conception that must be abandoned from psychometrics in favor of latent variables. First, we reiterate that the sum score stochastically orders the latent variable in a wide variety of much-used item response models. In fact, item response theory provides a mathematically based justification for the ordinal use of the sum score. Second, because discussions about the sum score often involve its reliability and estimation methods as well, we show that, based on very general assumptions, classical test theory provides a family of lower bounds several of which are close to the true reliability under reasonable conditions. Finally, we argue that eventually sum scores derive their value from the degree to which they enable predicting practically relevant events and behaviors. None of our discussion is meant to discredit modern measurement models; they have their own merits unattainable for classical test theory, but the latter model provides impressive contributions to psychometrics based on very few assumptions that seem to have become obscured in the past few decades. Their generality and practical usefulness add to the accomplishments of more recent approaches.

show abstract

In models we trust: preregistration, large samples, and replication may not suffice

Spiess,

Jordan

2023

Front. Psychol.

View full text Add to dashboard Cite

Despite discussions about the replicability of findings in psychological research, two issues have been largely ignored: selection mechanisms and model assumptions. Both topics address the same fundamental question: Does the chosen statistical analysis tool adequately model the data generation process? In this article, we address both issues and show, in a first step, that in the face of selective samples and contrary to common practice, the validity of inferences, even when based on experimental designs, can be claimed without further justification and adaptation of standard methods only in very specific situations. We then broaden our perspective to discuss consequences of violated assumptions in linear models in the context of psychological research in general and in generalized linear mixed models as used in item response theory. These types of misspecification are oftentimes ignored in the psychological research literature. It is emphasized that the above problems cannot be overcome by strategies such as preregistration, large samples, replications, or a ban on testing null hypotheses. To avoid biased conclusions, we briefly discuss tools such as model diagnostics, statistical methods to compensate for selectivity and semi- or non-parametric estimation. At a more fundamental level, however, a twofold strategy seems indispensable: (1) iterative, cumulative theory development based on statistical methods with theoretically justified assumptions, and (2) empirical research on variables that affect (self-) selection into the observed part of the sample and the use of this information to compensate for selectivity.

show abstract

Incomplete Tests of Conditional Association for the Assessment of Model Assumptions

Cited by 3 publications

References 85 publications

A Test to Distinguish Monotone Homogeneity from Monotone Multifactor Models

A Test to Distinguish Monotone Homogeneity from Monotone Multifactor Models

Recognize the Value of the Sum Score, Psychometrics’ Greatest Accomplishment

In models we trust: preregistration, large samples, and replication may not suffice

Contact Info

Product

Resources

About