Evaluating the response effort and data quality of established political solidarity measures: a pre-registered experimental test in an online survey of the German adult resident population in 2021

Abstract

This experimental study aims to check and improve the quality of 16 established survey measures of political solidarities and related concepts, such as redistribution and social trust. Political solidarities are defined as one’s willingness to share the costs that result from public redistribution that favours people other than oneself and thus constitute a subset of welfare state attitudes. The pre-registered study plan included suggestions for the development of improved rating scales, which we defined as five-point, end verbalized rating scales without non-substantive answer options. The overall results from an experimental online survey in Germany indicate differences in response effort in terms of response times but almost no differences in data quality in terms of criterion validity. Thus, the 16 survey measures show solid instrument validity as well as minor improvements in respondents’ response times. Indeed, the measures are (at least) in the online survey world of Germany of high-quality and warrant inclusion in future surveys with small efficiency gains still attainable.

Question order effects: how robust are survey measures on political solidarities with reference to Germany and Europe?

“…Finally, in this study, we only employed two questions on political solidarities that concerned redistribution. However, political solidarities are a multi-dimensional concept (Goerres & Höhne, 2023 ) and thus we recommend to consider a more diverse set of questions.…”

“…Measuring political solidarities – i.e. the individual willingness to share costs by public redistribution favoring other people than oneself (Goerres & Höhne, 2023 ) – is key in political science research and many adjacent research fields to infer public opinion. Accordingly, many established social surveys, such as the Eurobarometer and European Social Survey (ESS), frequently employ survey questions on political solidarities and related concepts, such as redistribution and welfare chauvinism.…”