A Regression Perspective on Generalized Distance Covariance and the Hilbert–Schmidt Independence Criterion

Abstract

In a seminal paper, Sejdinovic et al. (Ann. Statist. 41 (2013) 2263-2291 showed the equivalence of the Hilbert-Schmidt Independence Criterion (HSIC) and a generalization of distance covariance. In this paper, the two notions of dependence are unified with a third prominent concept for independence testing, the "global test" introduced in (J. R. Stat. Soc. Ser. B. Stat. Methodol. 68 (2006) 477-493). The new viewpoint provides novel insights into all three test traditions, as well as a unified overall view of the way all three tests contrast with classical association tests. As our main result, a regression perspective on HSIC and generalized distance covariance is obtained, allowing such tests to be used with nuisance covariates or for survival data. Several more examples of cross-fertilization of the three traditions are provided, involving theoretical results and novel methodology. To illustrate the difference between classical statistical tests and the unified HSIC/distance covariance/global tests we investigate the case of association between two categorical variables in depth.

Beyond Pearson’s Correlation: Modern Nonparametric Independence Tests for Psychological Research

“…The traditional method for doing this is to specify a statistical model that captures the desired relationship, such as a polynomial regression model. However, the findings in Edelmann and Goeman (2022) suggest that utilizing the Hilbert-Schmidt Independence Criterion (HSIC) with a kernel tailored to the anticipated associations may prove to be a more effective approach.…”

Handling Cellwise Outliers by Sparse Regression and Robust Covariance

“…In our opinion this topic would be a valuable addition to a graduate course on mathematical statistics, because distance covariance is a general method with interesting properties and wide ranging applications, for instance in variable selection (Chen et al, 2018), sparse contingency tables (Zhang, 2019), independent component analysis (Matteson and Tsay, 2017), and time series (Davis et al, 2018). It can be computed fast (Huo and Székely, 2016;Chaudhuri and Hu, 2019), and there are interesting connections with other dependence measures (Edelmann and Goeman, 2022). Its robustness to outliers was studied recently (Leyder et al, 2024).…”