Independent mechanism analysis, a new concept?

Abstract

Independent component analysis provides a principled framework for unsupervised representation learning, with solid theory on the identifiability of the latent code that generated the data, given only observations of mixtures thereof. Unfortunately, when the mixing is nonlinear, the model is provably nonidentifiable, since statistical independence alone does not sufficiently constrain the problem. Identifiability can be recovered in settings where additional, typically observed variables are included in the generative process. We investigate an alternative path and consider instead including assumptions reflecting the principle of independent causal mechanisms exploited in the field of causality. Specifically, our approach is motivated by thinking of each source as independently influencing the mixing process. This gives rise to a framework which we term independent mechanism analysis. We provide theoretical and empirical evidence that our approach circumvents a number of nonidentifiability issues arising in nonlinear blind source separation. * Equal contribution.Preprint. Under review.

Properties from Mechanisms: An Equivariance Perspective on Identifiable Representation Learning

“…In another line of work Locatello et al (2020); Shu et al (2019), the authors study the role of weak supervision in assisting disetanglement. In a recent work, Gresele et al (2021), propose to add new form of constraints to non-linear ICA. The constraint is based on the observation that the decoder g that gives rise to the image x is composed of simpler functions that are mutually algorithmically independent; the authors exploit this inductive bias on the structure of g to invert the data generation process.…”