An empirical evaluation of a novel domain-specific language – modelling vehicle routing problems with Athos

Abstract

Domain-specific languages (DSLs) are a popular approach among software engineers who demand for a tailored development interface. A DSL-based approach allows to encapsulate the intricacies of the target platform in transformations that turn DSL models into executable software code. Often, DSLs are even claimed to reduce development complexity to a level that allows them to be successfully applied by domain-experts with limited programming knowledge. Recent research has produced some scientifically backed insights on the benefits and limitations of DSLs. Further empirical studies are required to build a sufficient body of knowledge from which support for different claims related to DSLs can be derived. In this research study, we adopt current DSL evaluation approaches to investigate potential gains in terms of effectiveness and efficiency, through the application of our DSL Athos, a language developed for the domain of traffic and transportation simulation and optimisation. We compare Athos to the alternative of using an application library defined within a general-purpose language (GPL). We specified two sets of structurally identical tasks from the domain of vehicle routing problems and asked study groups with differing levels of programming knowledge to solve the tasks with the two approaches. The results show that inexperienced participants achieved considerable gains in effectiveness and efficiency with the usage of Athos DSL. Though hinting at Athos being the more efficient approach, the results were less distinct for more experienced programmers. The vast majority of participants stated to prefer working with Athos over the usage of the presented GPL’s API.

The Impact of Code Bloat on Genetic Program Comprehension: Replication of a Controlled Experiment on Semantic Inference

“…It inspired others to test similar hypotheses using different DSLs, different control and treatment groups, and different assignments. Studies (e.g., [50,[58][59][60]) showed that original hypotheses [49] can be generalized. Indeed, the comprehension correctness and efficiency of DSLs are better than of GPLs.…”

Tara: Streamlining DSL Development through Syntactic Patterns

“…DSLs are being used more and more in software development [4][5] [6], as they allow developers to express domain specific concepts in a more natural and easy to understand way. As a result, productivity is increased [7] [8],expressed in the domain concepts in a concise, natural way, facilitating the fulfilment of requirements, error detection, and promotion of changes with greater agility. However, the development of DSLs often requires a significant amount of time and effort, especially when creating a specialized language from scratch [9] [10].…”

Evolution of Domain-Specific Modeling Language: An Example of an Industrial Case Study on an RT-Sequencer

“…DSLs focus on domain abstractions, which empower end-users to program directly with concepts from the problem domain [4,5]. The benefits of using DSLs are increased flexibility, productivity, reliability, and usability, which have been demonstrated in empirical studies [6][7][8].…”