Vectorization technology to improve interpreter performance

Abstract

In the present computing landscape, interpreters are in use in a wide range of systems. Recent trends in consumer electronics have created a new category of portable, lightweight software applications. Typically, these applications have fast development cycles and short life spans. They run on a wide range of systems and are deployed in a target independent bytecode format over Internet and cellular networks. Their authors are untrusted third-party vendors, and they are executed in secure managed runtimes or virtual machines. Furthermore, due to security policies or development time constraints, these virtual machines often lack just-in-time compilers and rely on interpreted execution. At the other end of the spectrum, interpreters are also a reality in the field of high-performance computations because of the flexibility they provide.The main performance penalty in interpreters arises from instruction dispatch. Each bytecode requires a minimum number of machine instructions to be executed. In this work, we introduce a novel approach for interpreter optimization that reduces instruction dispatch thanks to vectorization technology. We extend the split compilation paradigm to interpreters, thus guaranteeing that our approach exhibits almost no overhead at runtime. We take advantage of the vast research in vectorization and its presence in modern compilers. Complex analyses are performed ahead of time, and their results are conveyed to the executable bytecode. At runtime, the interpreter retrieves this additional information to build the SIMD IR (intermediate representation) instructions that carry the vector semantics. The bytecode language remains unmodified, making this representation compatible with legacy interpreters and previously proposed JIT compilers.We show that this approach drastically reduces the number of instructions to interpret and decreases execution time of vectorizable applications. Moreover, we map SIMD IR instructions to hardware SIMD instructions when available, with a substantial additional improvement. Finally, we finely analyze the impact of our extension on the behavior of the caches and branch predictors.

On-the-fly Operation Batching in Dynamic Computation Graphs

“…Related to this is the general technique of automatic vectorization, which is a mainstay of optimizing compilers. Recent work has begun to explore vectorization in the context of interpreted code which may cannot be compiled [25]. Our autobatching variant of DyNet similarly provides vectorized primitives that can be selected dynamically.…”

Multi-Processor System-on-Chip Prototyping Using Dynamic Binary Translation

“…. , AVX for Intel) is huge, so having legacy code being able to benefit from the most efficient version available on the actually running processor has a clear value for some applications [26].…”

Tracing Flow Information for Tighter WCET Estimation: Application to Vectorization

“…Vectorization is a compiler optimization that transforms a scalar implementation of a computation into a vector implementation [4], [5]. It consists in processing multiple data at once, instead of processing a single data at a time.…”

“…Gcc distributes a set of loops collected on the GCC vectorizer example page 4 . It is now also one of the LLVM test suites and we can test it in LLVM compiler, 5 . We restricted our experiments to the 15 single-path programs in this benchmark suite.…”