Language virtualization for heterogeneous parallel computing

Abstract

As heterogeneous parallel systems become dominant, application developers are being forced to turn to an incompatible mix of low level programming models (e.g. OpenMP, MPI, CUDA, OpenCL). However, these models do little to shield developers from the difficult problems of parallelization, data decomposition and machine-specific details. Most programmers are having a difficult time using these programming models effectively. To provide a programming model that addresses the productivity and performance requirements for the average programmer, we explore a domainspecific approach to heterogeneous parallel programming.We propose language virtualization as a new principle that enables the construction of highly efficient parallel domain specific languages that are embedded in a common host language. We define criteria for language virtualization and present techniques to achieve them. We present two concrete case studies of domain-specific languages that are implemented using our virtualization approach.

Declarative Language Design for Interactive Visualization

“…For example, C# now supports anonymous (lambda) functions and enables declarative data manipulation via the LINQ language extension. The Scala language combines functional and object-oriented paradigms, can target both the JVM and the .NET CLR, and is being extended to support staged compilation via "lifting" of program components [2]. This enables developers to insert customized routines for manipulating a program's abstract syntax tree and performing code generation.…”

“…By implementing a DSL for visualization within a host programming language, designers can use familiar syntactic constructs, leverage the capabilities of the host language, and integrate with other projects. Already, this approach has proven popular in the functional programming community (e.g., [5,12]) and is used to facilitate massively parallel programming [2,17].…”

Scala-Virtualized: linguistic reuse for deep embeddings

“…The concept of linguistic reuse goes back to Krishnamurthi [26]. Language virtualization was introduced by Chafi et al [9]. The idea of representing an embedded language abstractly as a set of available methods was introduced as finally tagless embedding by Carette et al [8] and as polymorphic embedding by Hofer et al [20], going back to much earlier work by Reynolds [39].…”

“…The concept of language virtualization [9] is an attempt to capture the goal of reducing this trade-off in an analogy to the hardware world: Like in a data center, where one wants to virtualize costly "big iron" server resources and run many logical machines on top of them, it is desirable to leverage the engineering effort that went into a general-purpose language to support many small embedded languages, each of which should behave more or less like a real language, ideally according to all the different criteria given above.…”

JavaScript as an Embedded DSL

“…Previous work has established LMS as a pragmatic approach to runtime code generation and compiled DSLs. In particular, the Delite framework [4,29,7] uses this approach to provide an extensible suite of high-performance DSLs targeting heterogeneous parallel platforms (with options to generate code to Scala, C and Cuda) [21], for domains such as machine learning [33], numeric array processing [35] and mesh-based partial differential equation solvers [6]. LMS has also been used to generate SQL queries [37].…”