Post-Estimation Smoothing: A Simple Baseline for Learning with Side Information

Abstract

Observational data are often accompanied by natural structural indices, such as time stamps or geographic locations, which are meaningful to prediction tasks but are often discarded. We leverage semantically meaningful indexing data while ensuring robustness to potentially uninformative or misleading indices. We propose a post-estimation smoothing operator as a fast and effective method for incorporating structural index data into prediction. Because the smoothing step is separate from the original predictor, it applies to a broad class of machine learning tasks, with no need to retrain models. Our theoretical analysis details simple conditions under which post-estimation smoothing will improve accuracy over that of the original predictor. Our experiments on large scale spatial and temporal datasets highlight the speed and accuracy of post-estimation smoothing in practice. Together, these results illuminate a novel way to consider and incorporate the natural structure of index variables in machine learning.

Random Spatial Forests

“…However, some studies suggest that applying random forests and other machine learning methods, which do not utilize spatial information, to spatial data do not yield any noticeable advantages over traditional geostatistical approaches such as kriging (Berrocal et al 2019, Fox et al 2018. In order to correct for this deficiency, some have proposed a two-step approach applying kriging or smoothing of the residuals from the statistical learning technique in attempt to add spatial structure into predictions (Rolf et al 2020). In many cases, this two-step approach has been shown to perform better than either using either method alone (Liu et al 2018).…”