Facilitating Test-Driven Development via Domain-Specific Languages in Computational Science Software Engineering

Abstract

Research software plays a critical role in computational science. Therefore, the software should be of high quality. Producing such software involves efforts not only in initial development, but also during maintenance. One concept for improving the quality of software and reducing maintenance efforts is Test-Driven Development (TDD), which involves implementing tests, e.g., unit tests, as a specification for development. The effects are high test coverage and better overall software quality, but also additional effort in implementing tests. By separation of concerns, such additional efforts can be facilitated through the use of tailored notations such as domain-specific languages (DSLs). DSLs provide different levels of abstraction, for example, to separate the definition of tests from other concerns in software development. Not only does this incentivize research software engineers (RSEs) to write tests, but it also helps reduce the time required to write tests. The DSL TDD-DSL addresses this to facilitate the implementation of TDD using the Fortran Unit Testing Framework (UTF) pFUnit 4.0. TDD-DSL is focused on unit test definition and source code integration. It is implemented in Python for system under tests (SUTs) written in Fortran 90 or later. To reduce the effort of defining tests, TDD-DSL separates the test definition from the implementation and setup. Especially the latter can cause additional effort in practice, which can be mitigated by using code generators provided with the DSL tooling. The DSL allows describing test conditions as assertions and test cases as collections of assertions. TDD-DSL provides content assist within code editors for the underlying UTF, reducing the effort required in its deployment. It also supports parsing the SUT to expand the content assist with source code information and high-level error messaging for the test description. The content assist includes publicly available variables, types, functions and routines. TDD-DSL further supports the definition of unit values such as SI units to help Scientific Modelers check unit values in the software at runtime. For this purpose, the DSL tooling enables the generation of code templates for new implementations in the source code according to the defined tests. To simplify extending the DSL and improve overall maintainability, the DSL tooling employs the language server protocol (LSP) and also public available tools such as the ANTLR4 grammar. Using LSP also enables a multi-editor support to integrate the DSL tooling into different editors and IDEs. In this paper we report on the design and implementation of the DSL and its tooling as well as the evaluation with Scientific Modelers and RSEs.