Scalable Gaussian Processes for Data-Driven Design Using Big Data With Categorical Factors

Abstract

Scientific and engineering problems often require the use of artificial intelligence to aid understanding and the search for promising designs. While Gaussian processes (GP) stand out as easy-to-use and interpretable learners, they have difficulties in accommodating big datasets, qualitative inputs, and multi-type responses obtained from different simulators, which has become a common challenge for data-driven design applications. In this paper, we propose a GP model that utilizes latent variables and functions obtained through variational inference to address the aforementioned challenges simultaneously. The method is built upon the latent variable Gaussian process (LVGP) model where qualitative factors are mapped into a continuous latent space to enable GP modeling of mixed-variable datasets. By extending variational inference to LVGP models, the large training dataset is replaced by a small set of inducing points to address the scalability issue. Output response vectors are represented by a linear combination of independent latent functions, forming a flexible kernel structure to handle multi-type responses. Comparative studies demonstrate that the proposed method scales well for large datasets, while outperforming state-of-the-art machine learning methods without requiring much hyperparameter tuning. In addition, an interpretable latent space is obtained to draw insights into the effect of qualitative factors, such as those associated with “building blocks” of architectures and element choices in metamaterial and materials design. Our approach is demonstrated for machine learning of ternary oxide materials and topology optimization of a multiscale compliant mechanism with aperiodic microstructures and multiple materials.

Data‐Driven Design for Metamaterials and Multiscale Systems: A Review

“…This also limits the dimensionality of design problems to be considered, because the required amount of data scales exponentially with the dimension due to the curse of dimensionality. Past work proposed ways to make GP more scalable with larger datasets, [95,[214][215][216][217][218] which can potentially expand the use cases of GP to larger datasets and higher problem dimensions in metamaterials design.…”

“…Reproduced with permission. [ 95,149 ] top) Copyright 2022, ASME. bottom) Copyright 2022, Elsevier Ltd. b) Multiscale design with unit cells that allow a smooth transition between two classes.…”

“…To address these issues, Wang et al. [ 87,95,149 ] proposed a multi‐response latent variable Gaussian process (MR‐LVGP) and its enhanced variants to transform the discrete classes into continuous 2D latent variables through statistical inference. The constructed latent space captured the effects of classes on the mechanical properties, which induced an interpretable distance metric that reflects the similarity with respect to properties (Figure 21a).…”

Uncertainty-aware mixed-variable machine learning for materials design

“…The latent variable configuration in the latent space also indicates the effects of different levels of a categorical variable on the response, thus making the model interpretable 32 . There are extensions of LVGP 34,35 that allow utilizing large training data, physical knowledge, as well as kernels other than RBF that are suitable for fitting functions with different characteristics. In this work, the vanilla LVGP is used in comparative studies.…”

Uncertainty-Aware Mixed-Variable Machine Learning for Materials Design

“…The latent variable configuration in the latent space also indicates the effects of different levels of a categorical variable on the response, thus making the model interpretable 35 . There are extensions of LVGP 37 that allow utilising large training data, as well as kernels other than RBF that are suitable for fitting functions with different characteristics. In this work, the vanilla LVGP is used in comparative studies.…”