Multivariate conditional transformation models

Abstract

Regression models describing the joint distribution of multivariate responses conditional on covariate information have become an important aspect of contemporary regression analysis. However, a limitation of such models are the rather simplistic assumptions often made, for example, a constant dependence structure not varying with covariates or the restriction to linear dependence between the responses. We propose a general framework for multivariate conditional transformation models that overcomes these limitations and describes the entire distribution in a tractable and interpretable yet flexible way conditional on nonlinear effects of covariates. The framework can be embedded into likelihood-based inference, including results on asymptotic normality, and allows the dependence structure to vary with covariates. In addition, it scales well-beyond bivariate response situations, which were the main focus of most earlier investigations. We illustrate the benefits in a trivariate analysis of childhood undernutrition and demonstrate empirically that complex truly multivariate data-generating processes can be inferred from observations.

Count transformation models

“…The transformation approach seems also promising for the development for multivariate species distribution models, because different marginal transformation models can be combined into a multivariate model on the same scale (the idea was developed for continuous responses by Klein, Hothorn, & Kneib, 2019 and recent research focuses on discrete or count variables).…”

Bernstein Flows for Flexible Posteriors in Variational Bayes

“…TMs were also used for discrete outcome distribution, but here Bernstein polynomials are not suitable because the transformation function needs to be discrete [18], [20], [21]. While TMs have the focus on regression models [16] they were also successfully used for modeling unconditional and high dimensional distributions [22]. Also in NN models, TMs based on Bernstein polynomials have been used to model complex one-dimensional conditional distributions [21], [23], [24] or unconditional multidimensional distributions [25].…”

Distributional Gradient Boosting Machines

“…Hence, a future extension of our approach would be to directly model longitudinal or temporal dependencies as part of the training process, as discussed in, e.g., Sela and Simonoff (2012); Hajjem et al (2011). Another interesting extension of distributional modelling, as proposed by O' Malley et al (2021); Klein et al (2022); Marra and Radice (2017); Klein and Kneib (2016), is to extend the univariate case to a multiple response setting, with several responses of interest that are potentially interdependent. Also, since our framework relies on multiparameter optimization, where a separate tree is grown for each parameter, estimating many parameters for a large dataset can become computationally expensive, especially for NFBoost, where M Bernstein-Polynomials in addition to 4 scale and shift parameters need to be estimated.…”

“…Klein et al (2022) andSick et al (2021) highlight the close resemblance between Normalizing Flows and Conditional Transformation Models, even though and in contrast to the initial formulation of Conditional Transformation Models, Normalizing Flows usually consist of several chained transformations.…”