Variable Selection and Data Quality Challenges in Impact Assessments

Abstract

The research is focused on the role of two related key concepts, namely variables and data, in the impact evaluations of public projects. A difficult task of the evaluators and researchers is to select the appropriate variables to ensure the best model of reality and satisfy the evaluation methods' needs. Therefore, the paper aims to look at the current knowledge and discuss how variables and data could be best used to connect the evaluation models, the particularities of the intervention with the potential of the advanced quantitative assessment methods. The results emphasise that evaluations operate with data with different levels of granularity, as required by the intervention logic. Structuring data in clusters and categories, performing evaluability assessments are useful in assessing data quality and limitations and improving them. In line with the existing literature, we demonstrate that data accessibility is a key constraint and imposes adjustment of the desired evaluation model to a feasible one. While Big Data and Open Data systems significantly improved data quality in evaluations in recent years, blockchain, as a ledger technology with default features related to decentralisation and security, is expected to bring large benefits to evaluation. For evaluators and policymakers, blockchain potential is an area of further research looking for additional advantages that could enhance the use of quantitative methods.