Scalable Gaussian Processes, GPyTorch Application Benchmarking, and Targeted Adaptive Design (TAD) on ThetaGPU

Abstract

We aim at showcasing the scalability of a Gaussian process (GP). The naive GP implementation scales cubically with data size, which can be prohibitive, so GP has not heretofore been considered suitable for very large-scale problem settings. We take advantage of GPyTorch, a library for scalable GPs built on top of PyTorch, that incorporates GPU acceleration. With GPyTorch, one can achieve nearly linear scaling with structured kernel interpolation and constant-time predictive covariance computation with LanczOs Variance Estimates (LOVE) while preserving accuracy. We also take advantage of the computational power of ThetaGPU, a supercomputer in the Argonne Leadership Computing Facility. In addition, we implement a scalable, GPU-ready version of targeted adaptive design (TAD), a GP-based data-driven algorithm that efficiently searches the control space of an advanced manufacturing experiment for settings capable of producing a required design within a specified tolerance, despite the poorly known mapping from control settings to design. We show our benchmarking for GPyTorch and TAD performance on a CPU vs. ThetaGPU and discuss the results and implications.