Probability of bivariate superiority: A non-parametric common-language statistic for detecting bivariate relationships

Abstract

Researchers often focus on bivariate normal correlation (r) to evaluate bivariate relationships. However, these techniques assume linearity and depend on parametric assumptions. We propose a new nonparametric statistical model that can be more intuitively understood than the conventional r: probability of bivariate superiority (PBS). Our development of B, the estimator of a PBS relationship, extends Dunlap's (1994) common-language transformation of r (CL) by providing a method to directly estimate PBS-the probability that when x is above (or below) the mean of all X, its paired y score will also be above (or below) the mean of all Y. Probability of superiority is an important form of bivariate relationship that until now could only be accurately estimated when data met the parametric assumptions for r. We specify the copula that forms the theoretical basis for PBS, provide an algorithm for estimating PBS from a sample, and describe the results of a Monte Carlo experiment that evaluated our algorithm across 448 data conditions. The PBS estimate, B, is robust to violations of parametric assumptions and offers a useful method for evaluating the significance of probability-of-superiority relationships in bivariate data. It is critical to note that B estimates a different form of bivariate relationship than does r. Our working examples show that a PBS effect can be significant in the absence of a significant correlation, and vice versa. In addition to utilizing the PBS model in future research, we suggest that this new statistical procedure be used to find theoretically important but previously overlooked effects from past studies.

When is the Wilcoxon–Mann–Whitney procedure a test of location? Implications for effect‐size measures

“…The logic underlying the use of the Probability of Superiority as an effect size has been extended to bivariate correlational relationships in Dunlap () and more thoroughly in Li and Waisman () who develop the probability of bivariate superiority. Li () has extended it to the multivariate case for use in structural equation modelling.…”

Analytic and bootstrap confidence intervals for the common-language effect size estimate

“…In light of this, researchers have explored and considered alternative ESs beyond the d-family and r-family. On the basis of the probability-of-bivariate-superiority (PBS) theo ry, researchers (e.g., Cliff, 1993Cliff, , 1994Cliff, , 1996Cliff & Keats, 2003;McGraw & Wong, 1992;Li, 2016Li, , 2018aLi & Waisman, 2019;Ruscio, 2008) proposed the idea of common-lan guage effect size (CLES), which is regarded as a more understandable and interpretable ES than r and d. For example, instead of saying that there is a d (standardized mean difference) of 1.00 on a cognitive ability test between the treatment group and control group, a researcher can express that there is a 76% likelihood (CLES = Φ d / 2 , where Φ is the normal cumulative distribution function, and data are assumed to follow normal distribution; Ruscio, 2008) that a randomly-selected treatment group participant will perform better on a cognitive ability test than a randomly-selected control group partici pant.…”

“…For example, instead of saying that 16% (r = .4, or r 2 = .16) of variance of sons' heights is explained by variance in their fathers' heights, one can state that "a father who is above average in height has a 63% likelihood of having a son of above-average height" (Dunlap, 1994, p. 510). The mathematical proof for Dunlap's r-to-CL r conversion is shown in Li and Waisman (2019): if there is a linear correlation between normally distributed X and Y continuous scores, then the x-plane and y-plane can be divided into four quadrants based on the lines x = x and y = y (where x is the mean of X and y is the mean of Y). An observed correlation between X and Y (r) can then be converted to its corresponding CL r through Equation 1 on the basis of the number of sample observations (n 1 ) that belong to the first or third quadrants compared with the total number of observations (n).…”

“…Despite the potential of CL r , its use is relatively limited in practice because a) re searchers may perceive that Dunlap's (1994) CL r is merely a r-translated statistic useful for better knowledge mobilization only, and b) there is no analytic method for estimat ing the standard error (SE) and CI for this estimate. Indeed, Li and Waisman's (2019) study shows that the bivariate linear and normal correlation (BLNC) conditions are not necessary for researchers to obtain and interpret PBS. Instead, researchers can use and report the non-parametric version of CL r , which is known as B p in Li and Waisman.…”

“…Instead, researchers can use and report the non-parametric version of CL r , which is known as B p in Li and Waisman. This study aims to extend Blomqvist's (1950), and Li and Waisman's (2019) work by proposing and developing analytic methods (i.e., analytic-z and analytic-t) for estimating the SE and CI surrounding B p , which offers the necessary mathematical foundation for B p . This can be used by both theoretical researchers who are interested in further testing and generalizing B p to other data scenarios (e.g., multivariate relationships) and by applied researchers who are interested in evaluating their data based on B p , and comparing the performance of these methods with the empirical methods (i.e., bootstrap percentile interval [BPI], bootstrap bias-corrected-and-accelerated interval [BCaI], bootstrap stand ard interval [BSI] based on the empirical z distribution [BSI-z], and BSI based on the empirical t distribution [BSI-t] in Li & Waisman, 2019).…”