On-the-Fly Principled Speculation for FSM Parallelization

Abstract

Finite State Machine (FSM) is the backbone of an important class of applications in many domains. Its parallelization has been extremely difficult due to inherent strong dependences in the computation. Recently, principled speculation shows good promise to solve the problem. However, the reliance on offline training makes the approach inconvenient to adopt and hard to apply to many practical FSM applications, which often deal with a large variety of inputs different from training inputs.This work presents an assembly of techniques that completely remove the needs for offline training. The techniques include a set of theoretical results on inherent properties of FSMs, and two newly designed dynamic optimizations for efficient FSM characterization. The new techniques, for the first time, make principle speculation applicable on the fly, and enables swift, automatic configuration of speculative parallelizations to best suit a given FSM and its current input. They eliminate the fundamental barrier for practical adoption of principle speculation for FSM parallelization. Experiments show that the new techniques give significantly higher speedups for some difficult FSM applications in the presence of input changes.

Enabling scalability-sensitive speculative parallelization for FSM computations

“…Figure 2 shows the speedup curve for an FSM benchmark on a Xeon Phi machine with 256 cores. As the blue line shows, the actual speedup increases linearly at the beginning before reaching about 10 cores, which is confirmed by prior work [37,38]. However, the increase becomes non-linear thereafter and even starts dropping after about 30 cores.…”

“…Prior work [37,38] introduce metrics like state feasibility and expected convergence length to tune the design of the predictor. They are inadequate to accurately model the details of the nondeterministic behaviors of speculation, such as the distribution of misspeculation penalty across different chunks.…”

“…Without answering these questions, existing methods may not only suffer from suboptimal performance, but also waste computing resources that would be otherwise used for other computations. As illustrated by Figure 1, when executed on a Xeon Phi processor with 256 logical cores, existing methods [37,38] result in suboptimal speedup on two FSM benchmarks, up to nearly 5X performance degradation comparing to the optimal ones. It is also important to note that the optimal number of cores varies across different FSMs.…”

“…Unlike prior work that focus on either designing speculation techniques [38] or reducing profiling cost [37], this work aims to achieve the optimal performance gain for speculative FSM parallelization with an accurate scalability analysis.…”

“…The models integrate a probabilistic analysis to capture the non-deterministic behaviors of speculation and an offline sample-based conditional regression (SCR) technique to characterize the cost variation of misspeculation. Unlike existing FSM characterization [37,38] that requires to profile the convergence property for every pair of states, SCR only profiles state pairs that are more likely to appear in the actual speculative execution. Based on the probabilistic models and SCR, this work designs both architecture-independent scalability analysis and architecture-aware scalability analysis.…”