Evaluation criteria for co-creation processes in climate services - validation by practitioners

Abstract

National and international climate services support adaptation to climate change. Climate services should aim to deliver climate service products, which are useable, useful, accurate and suitable for users. The advancement of effective adaptation measures to climate change, therefore, requires continuous interactions between climate service providers and users of the products. Good practices for the design of climate services include transdisciplinary approaches and co-creation of climate service products. To assess the effectiveness of this co-creation endeavours, evaluation is crucial. The concept of formative evaluation aims at assessing the co-creation of science-based climate services during the product development process with the intention to reflect and readjust the ongoing process. Evaluation criteria for this formative evaluation were looked at in the project NorQuATrans (Normativity, Objectivity and Quality Assessment of Transdisciplinary Processes, https://www.hicss-hamburg.de/projects/NorQuATrans/index.php.en) doing accompanying research with the knowledge transfer project ADAPTER (ADAPT tERrestrial systems, https://adapter-projekt.org/). In ADAPTER, simulation-based climate service products have been developed together with practitioners from the agricultural sector. As a result of this cooperation, a formative evaluation scheme of 17 quality criteria for all the different sub-processes in climate service development was created, based on both empirical and theoretical work. To these 17 quality criteria, 72 related evaluation aspects were assigned and can be assessed by questions. In the following step we validated the formative evaluation scheme together with practitioners experienced in co-creation of climate services. For this validation we focused on the most important quality criteria to avoid overloading. After pretesting several notions and terms, a survey was organized. Based on the results of the survey, the evaluation scheme was adapted and extended with some more evaluation aspects. The poster will present results of the pretest and questionnaire and give some insights on differences in the attitude of practitioners and scientists towards co-creation and evaluation endeavours. Challenges of formative evaluation will be addressed.