The growth in evaluation practice has not always gone hand in hand with an increase in the actual quality of these evaluations. This article addresses this concern, by examining the factors that explain organizational variety in the application of evaluation quality assurance measures. We present the results of an analysis of 18 Flemish (Belgian) public sector organizations. To unravel different explanatory trajectories, we combine two comparative methods that rely on Boolean analysis: the most similar different outcome/most different similar outcome technique on the one hand, and crisp set qualitative comparative analysis on the other hand.