Speculation with Little Wasting: Saving Cost in Software Speculation through Transparent Learning

Abstract

Software speculation has shown promise in parallelizing programs with coarse-grained dynamic parallelism. However, most speculation systems use offline profiling for the selection of speculative regions. The mismatch with the inputsensitivity of dynamic parallelism may result in large numbers of speculation failures in many applications. Although with certain protection, the failed speculations may not hurt the basic efficiency of the application, the wasted computing resource (e.g. CPU time and power consumption) may severely degrade system throughput and efficiency. The importance of this issue continuously increases with the advent of multicore and parallelization in portable devices and multiprogramming environments.In this work, we propose the use of transparent statistical learning to make speculation cross-input adaptive. Across production runs of an application, the technique recognizes the patterns of the profitability of the speculative regions in the application and the relation between the profitability and program inputs. On a new run, the profitability of the regions are predicted accordingly and the speculations are switched on and off adaptively. The technique differs from previous techniques in that it requires no explicit training, but is able to adapt to changes in program inputs. It is applicable to both loop-level and functionlevel parallelism by learning across iterations and executions, permitting arbitrary depth of speculations. Its implementation in a recent software speculation system, namely the BehaviorOriented Parallelization system, shows substantial reduction of speculation cost with negligible decrease (sometimes, considerable increase) of parallel execution performance.

Tightfit: adaptive parallelization with foresight

“…Seminal behaviors are extracted automatically to support proactive optimizations in the Jikes VM. A similar approach, developed in [10], trades training for incremental adaptation across production runs. The authors of [10] apply this idea to predict the likelihood of successful speculation, where predictions account for input properties indirectly using classification trees.…”

“…A similar approach, developed in [10], trades training for incremental adaptation across production runs. The authors of [10] apply this idea to predict the likelihood of successful speculation, where predictions account for input properties indirectly using classification trees. Tightfit shares the motivation of these works, but supports direct learning of the relationship between input features and parallelism, rather than passing through seminal behaviors, thanks to offline learning.…”

Dynamic branch speculation in a speculative parallelization architecture for computer clusters

“…Speculation is based on the fact that the program behavior is usually repetitive and consequently predictable, as demonstrated in studies of branches , memory dependencies, and data values . Software speculation has recently shown promising results in parallelizing such programs . The relevant techniques can be classified into two types: Software speculation: Compilers carry out the necessary coding.…”