No abstract

No abstract

This article addresses two research questions related to reproducibility within the context of research related to computer science. First, a survey on reproducibility addressed to researchers in the academic and private sectors is described and evaluated. The survey indicates a strong need for open and easily accessible results, in particular, reproducing an experiment should not require too much effort. The results of the survey are then used to formulate guidelines for making research results reproducible. In addition, this article explores four approaches based on software tools that could bring forward reproducibility in research results. After a general analysis of tools, three examples are further investigated based on actual research projects which are used to evaluate previously introduced tools. Results indicate that the evaluated tools contribute well to making simulation results reproducible but due to conflicting requirements, none of the presented solutions fulfills all intended goals perfectly.

No abstract

This paper addresses two questions related to reproducibility within the context of research related to computer science. First, requirements on reproducibility are analyzed based on a survey addressed to researchers in the academic and private sector. The survey indicates a strong need for open but also easily accessible results, thus reproducing an experiment should not require too much effort. The results from the survey are then used to formulate general guidelines for making research results reproducible. In addition, this paper explores a number of existing software tools that could bring forward reproducibility in research results. After a general analysis of tools a further investigation is done via three case studies based on actual research projects which are used to evaluate the previously introduced tools. Results indicate that due to conflicting requirements, none of the presented solutions fulfills all intended goals perfectly. However, we present requirements and guidelines for making research reproducible. While the main focus of this paper is on reproducibility in computer science, the results of this paper are still valid for other fields using computation as a tool.