Characterising the Evolution of Energy System Models Using Model Archaeology

Abstract

In common with other types of complex models, energy system models have opaque structures, making it difficult to understand both changes between model versions and the extent of changes described in research papers. In this paper, we develop the principle of model archaeology as a formal method to quantitatively examine the balance and evolution of energy system models, through the ex post analysis of both model inputs and outputs using a series of metrics. These metrics help us to understand how models are developed and used and are a powerful tool for effectively targeting future model improvements. The usefulness of model archaeology is demonstrated in a case study examining the UK MARKAL model. We show how model development has been influenced by the interests of the UK government and the research projects funding model development. Despite these influences, there is clear evidence of a strategy to balance model complexity and accuracy when changes are made. We identify some important long-term trends including higher technology capital costs in subsequent model versions. Finally, we discuss how model archaeology can improve the transparency of research model studies.

Uncertainty analysis using Bayesian Model Averaging: a case study of input variables to energy models and inference to associated uncertainties of energy scenarios

“…The relevance of model changes (structural, system constraints and parameters) for model output is significant, cf. [33]. This indicates that a model-designed accordingly-may generate any output.…”

Addressing Energy System Modelling Challenges: The Contribution of the Open Energy Modelling Framework (oemof)

“…It has already been mentioned by other authors that the field of energy system modelling and its models may seem opaque to outsiders [22,23]. One reason for this may be the broad definition of the term energy system model.…”

“…Even if this is fulfilled, reproducibility remains a challenging task due to hardware, software and natural language uncertainty. The aspect of constantly changing model versions in energy system models and the lack of precisely describing these when presenting results adds another dimension to the challenge of reproducibility [22]. As Pfenninger et al [65] argue, full -meaning effective -transparency of energy system models is still hampered by different barriers.…”

Raising awareness in model-based energy scenario studies—a transparency checklist

“…It recommends four major documentation levels: (1) rote operation of the model, (2) model use, (3) model maintenance and (4) model assessment. Another documentation framework, especially designed for energy system models, is published by Dodds et al [37]. The focus of the work lies on the challenges due to the increasing complexity which is affected by the ongoing development of often applied optimization models.…”

“…In addition, usually model source code evolves continuously which demands maintaining also the documentation. In this regard, Dodds et al [37] propose an approach called model archaeology to especially incorporate the effect of different model versions on the documentation in a structured way. While open source represents the most detailed information level in this context, it can be stressed that even limited access to the models' source codes contributes to an increase in transparency of a certain ESS.…”