Investigating Ceiling Effects in Longitudinal Data Analysis

Abstract

Score limitation at the top of a scale is commonly termed "ceiling effect." Ceiling effects can lead to serious artifactual parameter estimates in most data analysis. This study examines the consequences of ceiling effects in longitudinal data analysis and investigates several methods of dealing with ceiling effects through Monte Carlo simulations and empirical data analyses. Data were simulated based on a latent growth curve model with T = 5 occasions. The proportion of the ceiling data [10%-40%] was manipulated by using different thresholds, and estimated parameters were examined for R = 500 replications. The results showed that ceiling effects led to incorrect model selection and biased parameter estimation (shape of the curve and magnitude of the changes) when regular growth curve models were applied. The Tobit growth curve model, instead, performed very well in dealing with ceiling effects in longitudinal data analysis. The Tobit growth curve model was then applied in an empirical cognitive aging study and the results were discussed.Many tests and scales have been developed in psychological and educational research to measure participants' abilities, well-being, and other constructs. However, if a test is relatively easy, high-scoring participants may answer every item correctly and reach the highest possible score, or ceiling, on the test. When this happens, the true extent of the high-scoring participants cannot be correctly measured, and this phenomenon is usually termed "ceiling effects." Uttl (2005) defined the ceiling effects as occurring when the tests or scales are relatively easy such that substantial proportions of individuals obtain either maximum or near-maximum scores and the true extent of their abilities cannot be determined.Ceiling effects are related to, but different from, performance asymptotes. Asymptotes occur when participants' scores cannot exceed a specific value with more information, additional practice, or retests (Miller, 1956). The asymptotic values are the greatest true values that participants can actually demonstrate. In this study, we assume ceiling effects happen before participants reach asymptotic values. The concept of ceiling effects is also distinct from the concept of semicontinuous variables (Olsen & Schafer, 2001). A semicontinuous variable combines a continuous distribution with point masses at one or more locations. For example, in alcohol usage research, the alcohol usage variable is a mixture of 0s and continuously distributed positive values, which is one of the typical semicontinuous variables. The difference between semicontinuous variables and ceiling effects is that the 0s are valid data values, not

Influence of regular soccer or swimming practice on gross motor development in childhood

“…In our opinion, this seems to be the most plausible reason for the decrease in both subtest standard scores (and in the gross motor quotient), between T10 and T30, when the raw score in most fundamental motor skills increases and reaches maximum values (or nearly that). This score limitation at the top of a scale is commonly termed "ceiling effect" (Wang, Zhang, McArdle & Salthouse, 2009). …”

Effects of a Strengths Intervention on General and Work-Related Well-Being: The Mediating Role of Positive Affect

“…In line with this assumption, it has been found that satisfaction with life could be enhanced through an extensive, 12-week strengths intervention (Rust et al 2009), but not through a relatively short, 3-week online intervention (Mitchell et al 2009). Next to the reasoning that more extensive interventions might be required, not finding direct effects of the strengths intervention on work-related well-being might also be partly explained by the participants' high initial levels of engagement and low levels of burnout which makes it more difficult to achieve and detect changes for the better (ceiling effect; Wang et al 2008). …”

Univariate and multivariate skewness and kurtosis for measuring nonnormality: Prevalence, influence and estimation

“…For example, high school GPA of students who apply for colleges often shows such a distribution because students with lower GPA are less likely to seek a college degree. In psychological research, scores on easy cognitive tasks tend to be negatively skewed because the majority of participants can complete most tasks successfully (Wang et al, 2008). Kurtosis is associated with the tail, shoulder and peakedness of a distribution.…”