R melts brains: an IR for first-class environments and lazy effectful arguments

Abstract

The R programming language combines a number of features considered hard to analyze and implement efficiently: dynamic typing, reflection, lazy evaluation, vectorized primitive types, first-class closures, and extensive use of native code. Additionally, variable scopes are reified at runtime as first-class environments. The combination of these features renders most static program analysis techniques impractical, and thus, compiler optimizations based on them ineffective. We present our work on PIR, an intermediate representation with explicit support for first-class environments and effectful lazy evaluation. We describe two dataflow analyses on PIR: the first enables reasoning about variables and their environments, and the second infers where arguments are evaluated. Leveraging their results, we show how to elide environment creation and inline functions.CCS Concepts • Software and its engineering → Compilers.

Promises are made to be broken: migrating R to strict semantics

“…The alternative was to use an experimental native compiler. There are two available in open source, Oracle's Truffle implementation of R [Stadler et al 2016], and our own Ř [Flückiger et al 2019]. Both systems have their drawbacks.…”

“…The first tier is realized by a bytecode interpreter, the second tier by an optimizing native compiler that relies on LLVM for code generation. The compiler employs, among many other optimizations, speculative inlining of R closures and promises [Flückiger et al 2019. Choosing Ř allows us to better evaluate the impact of laziness.…”

Formally verified speculation and deoptimization in a JIT compiler

“…The early V8 JavaScript engine lowered both baseline and optimized functions to native code, and deoptimization toggled between the two; as of now the baseline is an interpreter [Chromium 2020]. The R compiler of Flückiger et al [2019] performs speculation on a dedicated intermediate representation (IR) and offloads code generation to LLVM, with write-only code segments.…”

“…As in Graal [Duboscq et al 2013], deoptimization is specified at the IR level by dedicated instructions which have a formally specified semantics. Without loss of generality, we restrict our attention to a uniform execution model in which both the unoptimized and the optimized code are expressed in the same intermediate language, similarly to the approach adopted by Béra et al [2016] and Flückiger et al [2019]. Furthermore, we carefully separate mechanism, i.e., how code is generated, optimized and deoptimized, from policy, i.e., when and what to optimize.…”

“…This compiler architecture follows[Flückiger et al 2019] where R programs are JIT compiled by IR rewriting.Proc. ACM Program.…”