It’s all Greek to me: Explaining, computing, and summarizing traditional and (re)emerging metrics of reliability for seven measures in sexual science

Abstract

Sexual scientists typically default to appraising the reliability of their self-report measures by calculating one or more α coefficients. Despite the prolific use of α, few researchers understand how to situate and make sense of α within the psychometric theories used to develop the measures used in their research (e.g., latent variable theory) and many unknowingly violate the assumptions of α. In this paper, we describe the disconnect between α and latent variable theory and the subsequent restrictive assumptions α makes. Simultaneously, we introduce an alternative metric of reliability—omega (ɷ)—that is compatible with latent variable theory. Subsequently, we provide a tutorial to walk readers through didactic examples on how to calculate ɷ metrics of reliability using the getOmega() function—a simple open-source function we created to automate the estimation of ɷ. We then introduce the Measurement of Sexuality and Intimacy Constructs (MoSaIC) project to provide insight into the state of reliability in sexuality science. We do this through contrasting α and ɷ estimates of reliability across seven sexuality measures, selected based on their emerging and pre-existing relevance and influence in the field of sexuality, in both a queer (LGBTQ+) sample ( n = 545) and a United States’ representative sample ( n = 548). We finish our paper with pragmatic suggestions for editors, reviewers, and authors. By more deeply understanding one’s options of reliability metrics, sexual scientists may carefully consider how they present and assess their measures’ reliability, and ultimately help improve our science’s replicability.

Feeling Close and Seeing a Partner in a New Light: How Self-Expansion Is Associated With Sexual Desire

“…For estimates of reliability, we calculated omega rather than Cronbach’s alpha for measures with three or more items. Omega is a more accurate way to assess within-person reliability (Gauvin et al, 2021; Lane & Shrout, 2010) and represents the ratio of variance accounted for by a general factor to the test variance (Flora, 2020) (see Table 2 for correlations).…”