Metric Learning for Simulation Analytics

Abstract

The sample path generated by a stochastic simulation often exhibits significant variability within each replication, revealing periods of good and poor performance alike. As such, traditional summaries of aggregate performance measures overlook the more fine-grained insights into the operational system behavior. In this paper, we take a simulation analytics view of output analysis, turning to machine learning methods to uncover key insights from the dynamic sample path. We present a k nearest neighbors model on system state information to facilitate real-time predictions of a stochastic performance measure. This model is built on the premise of a system-specific measure of similarity between observations of the state, which we inform via metric learning. An evaluation of our approach is provided on a stochastic activity network and a wafer fabrication facility, both of which give us confidence in the ability of metric learning to provide interpretation and improved predictive performance.

Fourier trajectory analysis for system discrimination

“…Nelson (2016) suggested the need for the field of simulation analytics to place greater emphasis on the statistical analysis of the entire simulation data series, characterizing behavior over time, rather than solely on steady-state characterizations. Work in this area has focused on dynamically varying forecasts of system behavior, such as expected value of waiting time conditional on number-in-system or time of day, or indicator variables for expected system behavior, given current state, which is time-dependent and thus dynamic (Lin et al, 2019;Laidler et al, 2020). These statistics characterize behavior at an instant that changes dynamically over time: a dynamic statistic of static behavior.…”