Causal Matrix Completion

Abstract

Matrix completion is the study of recovering an underlying matrix from a sparse subset of noisy observations. Traditionally, it is assumed that the entries of the matrix are "missing completely at random" (MCAR), i.e., each entry is revealed at random, independent of everything else, with uniform probability. This is likely unrealistic due to the presence of "latent confounders", i.e., unobserved factors that determine both the entries of the underlying matrix and the missingness pattern in the observed matrix. For example, in the context of movie recommender systems-a canonical application for matrix completion-a user who vehemently dislikes horror films is unlikely to ever watch horror films. In general, these confounders yield "missing not at random" (MNAR) data, which can severely impact any inference procedure that does not correct for this bias.We develop a formal causal model for matrix completion through the language of potential outcomes, and provide novel identification arguments for a variety of causal estimands of interest. We design a procedure, which we call "synthetic nearest neighbors" (SNN), to estimate these causal estimands. We prove finite-sample consistency and asymptotic normality of our estimator. Our analysis also leads to new theoretical results for the matrix completion literature. In particular, we establish entry-wise, i.e., max-norm, finite-sample consistency and asymptotic normality results for matrix completion with MNAR data. As a special case, this also provides entry-wise bounds for matrix completion with MCAR data. Across simulated and real data, we demonstrate the efficacy of our proposed estimator.

Extrapolating missing antibody-virus measurements across serological studies

“…For example, we used the same code to analyze the six influenza studies (with rank between 6 and 9) and the HIV-1 Catnap dataset containing hundreds of studies (with rank 23). We also found that nuclear norm minimization performed better than other imputation approaches (e.g., mean imputation or kNN regression; Figure S12 and STAR Methods) and also outperformed a recently published method of causal matrix completion developed to specifically complete data with values that are missing in non-random (structured) patterns (Agarwal et al, 2021). That said, many other forms of matrix completion exist, and our approach could be further refined by incorporating side information such as virus sequence or antibody isotype (Radhakrishnan et al, 2022).…”

“…Moreover, we tested a recently-published method of causal matrix completion that was developed to specifically deal with scenarios in which data are not missing at random, and in which the availability of data may be correlated with the outcome of the experiment (Agarwal et al, 2021). This could be relevant, for instance, in matrix completion of the HIV-1 Catnap data through a given date, since measurements made by the community were not randomly chosen, but rather chosen to be informative for a specific study.…”

Optimal Recovery for Causal Inference

“…Recently, there has been a growing literature that approaches causal inference from a matrix completion perspective. Proposals include approximating the control unit matrix using nuclear-norm minimization (Athey et al, 2021), using singular value decomposition (Amjad et al, 2018), or by finding nearest neighbors (Agarwal et al, 2021) for missing entries of a matrix to best match control units and the treated unit of interest.…”

Causal Inference for Social and Engineering Systems

“…This problem can be reduced to matrix completion, where rows index users and columns index items; each missing user-item entry corresponds to the potential rating a user would give to that item had they rated it. To motivate the importance of studying the missingness mechanism, we showcase two experiments (details inAgarwal et al (2021b)), one with MCAR and the other with MNAR data in Figures 1.6a and 1.7a. We use three matrix completion algorithms to recover the distribution of true ratings given a subset of revealed ratings: (i) Universal singular value thresholding (USVT) a popular spectral based method; (ii) Softimpute (softImpute), a popular optimization based method; (iii) "synthetic nearest neighbors" (SNN), our proposed method.…”