Gaussian Process Panel Modeling—Machine Learning Inspired Analysis of Longitudinal Panel Data

Abstract

In this article, we extend the Bayesian nonparametric regression method Gaussian Process Regression to the analysis of longitudinal panel data. We call this new approach Gaussian Process Panel Modeling (GPPM). GPPM provides great flexibility because of the large number of models it can represent. It allows classical statistical inference as well as machine learning inspired predictive modeling. GPPM offers frequentist and Bayesian inference without the need to resort to Markov chain Monte Carlo-based approximations, which makes the approach exact and fast. GPPMs are defined using the kernel-language, which can express many traditional modeling approaches for longitudinal data, such as linear structural equation models, multilevel models, or state-space models but also various commonly used machine learning approaches. As a result, GPPM is uniquely able to represent hybrid models combining traditional parametric longitudinal models and nonparametric machine learning models. In the present paper, we introduce GPPM and illustrate its utility through theoretical arguments as well as simulated and empirical data.

Comparative Analysis of Statistical Regression Models for Prediction of Live Weight of Korean Cattle during Growth

“…Gaussian process regression (GPR) model extends the linear regression model described in Equation (1) to accommodate nonlinear relationships between the response variable y and the input vector x [25,26]. This is achieved by introducing a function ϕ(x) that transforms the input vector x into a new space [26]:…”

“…Hence, the prior distribution of the regression function for a finite input vector x is represented using a multivariate Gaussian distribution [26]. However, when the input vector x is infinite, representing it as a Gaussian distribution becomes impossible.…”

“…To fully describe the prior distribution of the regression function, we need to specify the distribution of the Gaussian process { f (x), x ∈ X }. This Gaussian process is described by a mean function and a kernel [26]. This is abbreviated as:…”

“…The training set is utilized to train the model with the objective of obtaining optimal predictions for input vectors not present in the training set. GPR directly computes the predictive distribution in a single step [26]. The predictive distribution signifies the distribution provided the model and training set for predictions on the testing set, denoted as f (X * |D) .…”

“…This approach enables us to represent the joint distribution of observations Y(X) and predictions f (X * ) as follows [26]:…”