Pycket: a tracing JIT for a functional language

Abstract

We present Pycket, a high-performance tracing JIT compiler for Racket. Pycket supports a wide variety of the sophisticated features in Racket such as contracts, continuations, classes, structures, dynamic binding, and more. On average, over a standard suite of benchmarks, Pycket outperforms existing compilers, both Racket's JIT and other highly-optimizing Scheme compilers. Further, Pycket provides much better performance for proxies than existing systems, dramatically reducing the overhead of contracts and gradual typing. We validate this claim with performance evaluation on multiple existing benchmark suites.The Pycket implementation is of independent interest as an application of the RPython meta-tracing framework (originally created for PyPy), which automatically generates tracing JIT compilers from interpreters. Prior work on meta-tracing focuses on bytecode interpreters, whereas Pycket is a high-level interpreter based on the CEK abstract machine and operates directly on abstract syntax trees. Pycket supports proper tail calls and first-class continuations. In the setting of a functional language, where recursion and higher-order functions are more prevalent than explicit loops, the most significant performance challenge for a tracing JIT is identifying which control flows constitute a loop-we discuss two strategies for identifying loops and measure their impact.

Live Multi-language Development and Runtime Environments

“…Further, we want to find out if we can integrate more languages into Squimera, especially languages based on programming paradigms other than object-orientation. There are many RPython-based implementations of various languages that can be used, for example for Prolog [4], Racket [5], Lisp [2], or SQL [9].…”

How to Architect a Query Compiler

“…Figure 4d shows our working example in ScaLite[Map, List]. Implementing a hash join is done in two phases: the first phase (lines [3][4][5][6][7][8][9][10][11][12] iterates over the elements of the first relation and builds a hash table which groups these elements based on their join key; in the second phase, the algorithm probes the elements of the second relation and iterates over the corresponding elements of the first relation using the constructed hash table.…”

“…), this DSL contains different join operators including semi-, anti-, and outer joins. 8 Transformations. As we discussed in Section 3.2, declarative and functional languages are not appropriate for performance tuning.…”

“…The reasons for using ANF can be summarized as follows: first, ANF simplifies data-flow analysis by allowing a single definition of variables [14], which has a great impact on simplifying optimizations that use data-flow information, such as CSE -indeed, it facilitates the mix of effectful and pure computations, and has been shown to make optimizing transformations and analyses easier to write [8]; second, both ANF and SSA preserve the natural organization of programs (called "direct-style"), making them more understandable than CPS (which uses "continuation-passing style", whereby functions call continuations instead of returning values [25]); third, as ANF is a direct-style representation of λ-calculus [25], there are very well-known frameworks for the analysis and verification of λ-calculus developed in the PL community [58]. Hence, reasoning about compiler optimizations in ANF is simpler [14]; finally, by converting many semantically equivalent programs into a canonical representation, expressing optimizations becomes simpler [49], as there is no more any need to express optimizations for all semantically equivalent cases: it suffices to only express one for the canonical form.…”

Approaches to interpreter composition

“…'(3, []))). Furthermore, a list can be denoted in terms of its head and tail, for example [1 | [2,3]] is equivalent to [1,2,3].…”