There are many tools available for the creation of domain specific languages (DSLs) but the question remains how to identify appropriate use cases for the application of domain specific modeling and language design, and how to measure success. We report on our observations after three years of accompanying several real-life industrial DSL design projects and on our experiments with applying qualitative and quantitative evaluation criteria. We suggest an evaluation methodology spanning the entire DSL life cycle. It consists of an assessment of motivation, qualitative interviews, a validation of DSL design, quantifying benefits and a comparison of impacted workflows before and after adoption. We conclude with a discussion of inherent limitations.

We observe that small and medium enterprises who wish to adopt domain specific modeling techniques do so under different preconditions and with different expectations. In our report, we categorize our observations made in 7 different industrial branches. Further, we present the current state of our solution to provide guidance to both ends of stakeholders involved in a DSM development lifecycle, domain experts and DSL designers. By supporting a DSL development process with semantic knowledge bases and metrics, our goal is to make a DSL development feasible and beneficial through the guidance provided by EXAMINE and within the DIESEL-Framework