From Non-preemptive to Preemptive Scheduling Using Synchronization Synthesis

Abstract

We present a computer-aided programming approach to concurrency. The approach allows programmers to program assuming a friendly, non-preemptive scheduler, and our synthesis procedure inserts synchronization to ensure that the final program works even with a preemptive scheduler. The correctness specification is implicit, inferred from the non-preemptive behavior. Let us consider sequences of calls that the program makes to an external interface. The specification requires that any such sequence produced under a preemptive scheduler should be included in the set of sequences produced under a nonpreemptive scheduler. We guarantee that our synthesis does not introduce deadlocks and that the synchronization inserted is optimal w.r.t. a given objective function. The solution is based on a finitary abstraction, an algorithm for bounded language inclusion modulo an independence relation, and generation of a set of global constraints over synchronization placements. Each model of the global constraints set corresponds to a correctness-ensuring synchronization placement. The placement that is optimal w.r.t. the given objective function is chosen as the synchronization solution. We apply the approach to device-driver programming, where the driver threads call the software interface of the device and the API provided by the operating system. Our experiments demonstrate that our synthesis method is precise and efficient. The implicit specification helped us find one concurrency bug previously missed when model-checking using an explicit, user-provided specification. We implemented objective functions for coarse-grained and fine-grained locking and observed that different synchronization placements are produced for our experiments, favoring a minimal number of synchronization operations or maximum concurrency, respectively.

Root Causing Linearizability Violations

“…Note that if τ is a non-linearizable trace, then all the traces τ that are equivalent to τ are also non-linearizable. Indeed, if τ is equivalent to τ , then the values that are read in τ are the same as in τ 6 , which implies that the return values in τ are the same as in τ , and therefore, τ is non-linearizable when τ is.…”

“…There is large body of work on synchronization synthesis [2,[6][7][8]11,22,34,42,43]. The approaches in [11,42] are based on inferring synchronization by constructing and exploring the entire product graph or tableaux corresponding to a concurrent program.…”

Resource-guided program synthesis

“…Synthesis with Quantitative Objectives Two lines of prior work on synthesis are explicitly concerned with optimizing resource usage. One is quantitative automata-theoretic synthesis, which has been used to synthesize optimal Mealy machines [8] and place synchronization in concurrent programs [11,12,28]. In contrast, we focus on synthesis of high-level programs that can manipulate custom data structures, which are out of reach for automata-theoretic synthesis.…”

“…One of the greatest challenges in software development is to write programs that are not only correct but also efficient with respect to memory usage, execution time, or domain specific resource metrics. For this reason, automatically optimizing program performance has long been a goal of synthesis, and several existing techniques tackle this problem for low-level straight-line code [9,48,49,55,56] or add efficient synchronization to concurrent programs [11,12,21,28]. However, the developed techniques are not applicable to recent advances in the synthesis of high-level looping or recursive programs manipulating custom data structures [22,35,39,46,50,51].…”

Abstract Learning Frameworks for Synthesis

“…We exhibit a variety of existing synthesis frameworks that can be naturally seen as instantiations of our abstract-learning framework, where the formulation shows the diversity in the instantiations of the spaces. These include (a) a variety of CEGIS-based synthesis techniques for synthesizing program expressions in sketches (completing program sketches [51], synthesizing loop-free programs [24], mining specifications [28], synthesizing synchronization code for concurrent programs [13], etc. ), (b) synthesis from input-output examples such as Flashfill [23], (c) the CEGIS framework applied to the concrete problem of solving synthesis problems expressed in the SMT-based SyGuS format [2,1], and three synthesis engines that use learning to synthesize solutions, (d) invariant synthesis frameworks, including Houdini [18] and the more recent ICE-learning model for synthesizing loop invariants [21], spanning a variety of domains from arithmetic [21,22] to quantified invariants over data structures [20], and (e) synthesizing fixed-points and abstract transformers in abstract interpretation settings [52].…”