Case-to-factor Ratios and Model Specification in Qualitative Comparative Analysis

Abstract

Qualitative comparative analysis (QCA) is an empirical research method that has gained some popularity in the social sciences. At the same time, the literature has long been convinced that QCA is prone to committing causal fallacies when confronted with non-causal data. More specifically, beyond a certain case-to-factor ratio, the method is believed to fail in recognizing real data. To reduce that risk, some authors have proposed benchmark tables that put a limit on the number of exogenous factors given a certain number of cases. Many applied researchers looking for methodological guidance have since adhered to these tables. We argue that fears of inferential breakdown in QCA due to an “unfavorable” case-to-factor ratio are without foundation. What is more, we demonstrate that these benchmarks induce more fallacious inferences than they prevent. For valid causal inference, researchers are better off relying on the current state of knowledge in their respective fields.

Fewer Procedures, More Reflection: A Rejoinder to Duşa and Marx

“…This problem has first been noted in a QCA software review by Thiem and Duşa (2013) and has since been extensively analyzed in Baumgartner and Ambühl (2020), Baumgartner and Thiem (2017), and Thiem et al (2020). In note 14 of Thiem and Mkrtchyan (2023), we pointed out that Muriaas et al (2022) have disallowed the identification of model ambiguity by activating row dominance and deactivating the identification of all models in the software they employed. With the two omitted factors included, but while keeping the (incorrect) software settings of row dominance and model suppression activated, 22 models result, all of which have exactly the same consistency and coverage scores.…”

“…Here, Duşa and Marx take issue with the set-up behind Table 1 in Thiem and Mkrtchyan (2023). In note 9 of Thiem and Mkrtchyan (2023), we provided the full explanation of why we let only one exogenous factor in that table vary, and not two, 10, or 20.…”

“…Here, Duşa and Marx take issue with the set-up behind Table 1 in Thiem and Mkrtchyan (2023). In note 9 of Thiem and Mkrtchyan (2023), we provided the full explanation of why we let only one exogenous factor in that table vary, and not two, 10, or 20. Duşa and Marx may have overlooked this note.(3) “For 30 causal conditions where only two cases are observed, there are more than a billion remainders to be employed […].…”

“…Contrary to our expectations, Duşa and Marx do not take issue with the core part of Thiem and Mkrtchyan (2023): the extensive simulations we have carried out and their results that have been summarized in Figure 2. Instead, their comment focuses on: (1) our failure to propose an enhancement of their benchmarks; (2) our use of constants in QCA; (3) our neglect of logical remainders; and (4) our reanalysis of the applied study by Muriaas et al (2022).…”

“…Here, Duşa and Marx again take issue with the set-up behind Table 1 in Thiem and Mkrtchyan (2023). This time, however, their criticism illustrates emblematically how disconnected the QCA community is from the very field from which it has once borrowed its algorithmic core.…”