Dynamically composing languages in a modular way: supporting C extensions for dynamic languages

Abstract

Many dynamic languages such as Ruby, Python and Perl offer some kind of functionality for writing parts of applications in a lowerlevel language such as C. These C extension modules are usually written against the API of an interpreter, which provides access to the higher-level language's internal data structures. Alternative implementations of the high-level languages often do not support such C extensions because implementing the same API as in the original implementations is complicated and limits performance.In this paper we describe a novel approach for modular composition of languages that allows dynamic languages to support C extensions through interpretation. We propose a flexible and reusable cross-language mechanism that allows composing multiple language interpreters, which run on the same VM and share the same form of intermediate representation -in this case abstract syntax trees. This mechanism allows us to efficiently exchange runtime data across different interpreters and also enables the dynamic compiler of the host VM to inline and optimize programs across multiple language boundaries. We evaluate our approach by composing a Ruby interpreter with a C interpreter. We run existing Ruby C extensions and show how our system executes combined Ruby and C modules on average over 3× faster than the conventional implementation of Ruby with native C extensions, and on average over 20× faster than an existing alternate Ruby implementation on the JVM (JRuby) calling compiled C extensions through a bridge interface. We demonstrate that cross-language inlining, which is not possible with native code, is performance-critical by showing how speedup is reduced by around 50% when it is disabled.

Applying Optimizations for Dynamically-typed Languages to Java

“…The C⁄C++ code is executed by Sulong [22], an implementation of C⁄C++ on top of Trule. The native interface itself is implemented using Trule's cross-language interoperability mechanism [13,14].…”

Memory-safe Execution of C on a Java VM

“…Now we will show how these objects and their members are accessed in the AST. This is done by sending messages to the managed allocations and by resolving these messages at run time [15,18]. All kinds of managed allocations are accessed in a uniform way.…”

“…Upon first execution, every message to an object resolves to an object-specific AST snippet (message resolution) [15,18], which is then inserted into the enclosing AST as a replacement for the message. In other words, message resolution replaces object-independent messages by objectspecific ASTs.…”

“…Object accesses via messages and message resolution are Truffle mechanisms that were already published in [15,18]. We use these techniques for accessing managed allocations.…”

Techniques and applications for guest-language safepoints

“…In the context of Truffle, some of this "native code" (e.g., Ruby C extensions running on top of TruffleC [5]) is actually also executed by the same VM. In this case, that execution is not considered to be native code by the VM and it is possible to add calls to poll() in that program like in the main guest-language implementation.…”