A Latent Variable Approach to Gaussian Process Modeling with Qualitative and Quantitative Factors

Abstract

Computer simulations often involve both qualitative and numerical inputs. Existing Gaussian process (GP) methods for handling this mainly assume a different response surface for each combination of levels of the qualitative factors and relate them via a multiresponse cross-covariance matrix. We introduce a substantially different approach that maps each qualitative factor to an underlying numerical latent variable (LV), with the mapped value for each level estimated similarly to the correlation parameters. This provides a parsimonious GP parameterization that treats qualitative factors the same as numerical variables and views them as effecting the response via similar physical mechanisms. This has strong physical justification, as the effects of a qualitative factor in any physics-based simulation model must always be due to some underlying numerical variables. Even when the underlying variables are many, sufficient dimension reduction arguments imply that their effects can be represented by a low-dimensional LV. This conjecture is supported by the superior predictive performance observed across a variety of examples. Moreover, the mapped LVs provide substantial insight into the nature and effects of the qualitative factors.

Bayesian Optimization for Materials Design with Mixed Quantitative and Qualitative Variables

“…The key idea of the LVGP approach is to map the qualitative factors into low-dimensional quantitative latent variable (LV) representations. Our study 28 showed that the LVGP modeling dramatically outperforms existing GP models with qualitative input factors in terms of predictive root mean squared error (RMSE). In addition to empirical evidence of far better RMSE predictive performance across a variety of examples, the LVGP approach has strong physical justification in that the effects of any qualitative factor on a quantitative response must always be due to some underlying quantitative physical input variables (otherwise, one cannot code the physics of the simulation model) 28 .…”

“…Our study 28 showed that the LVGP modeling dramatically outperforms existing GP models with qualitative input factors in terms of predictive root mean squared error (RMSE). In addition to empirical evidence of far better RMSE predictive performance across a variety of examples, the LVGP approach has strong physical justification in that the effects of any qualitative factor on a quantitative response must always be due to some underlying quantitative physical input variables (otherwise, one cannot code the physics of the simulation model) 28 . Noting that the underlying physical variables may be extremely highdimensional (which is why they are treated as a qualitative factor in the first place), the LVGP mapping serves as a low-dimensional LV surrogate for the high-dimensional physical variables that captures their collective effect on the response.…”

“…To overcome the aforementioned limitations, we propose a BO framework for data-driven materials design with mixed qualitative and quantitative variables ( Figure 2). The proposed framework is built upon a novel latent variable GP (LVGP) modeling approach that we recently developed for creating response surfaces with both qualitative and quantitative inputs 27 . The key idea of the LVGP approach is to map the qualitative factors into low-dimensional quantitative latent variable (LV) representations.…”

“…These types of correlation functions cannot be directly applied to problems with qualitative factors because the distances between levels of qualitative factors are not defined, and the levels have no natural ordering since are assumed to be nominal categorical factors.The LVGP approach28 provides a natural and convenient way to handle qualitative input variables by mapping the levels of each qualitative factor to a 2-dimensional (2D) continuous latent space. This has strong physical justification, which may explain the outstanding predictive performance of the LVGP approach28 : For any real physical system with a qualitative input factor , there are always underlying quantitative physical variables { % , D , … } = { % ( ), D ( ), … } (perhaps very high-dimensional, poorly understood, and difficult to treat individually as quantitative input variables) that account for the differences between the responses at different levels of the qualitative factors. As shown inFigure 8, the three qualitative levels are associated with points in a high-dimensional space of { % , D , … }, and the distances between the three points indicates the differences between the three levels.…”

Data-Centric Mixed-Variable Bayesian Optimization for Materials Design

“…The standard GP methods were developed under the premise that all input variables are quantitative, which does not hold in many real engineering applications. We recently proposed a latent variable Gaussian processes (LVGP) [25] modeling method that maps the levels of the qualitative factor(s) to a set of numerical values for some latent quantitative variable(s). The latent variables transform the underlying high dimensional physical attributes associated with the categorical variables into the latent-variable space and quantify the "distances" between samples.…”