Echoing the results of our primary investigation with the full survey, here we present results according to our reduced xAI survey. An ANCOVA showed that certain conditions in our experiment were rated as significantly more explainable than others (F (7, 277) = 3.14, p = 0.003). Our independent variable is the explainability method and our dependent variable is the explainability score. We include as a covariate the participant's baseline explainability score. A Shapiro-Wilk test revealed that our data were not normally distributed, but we proceed with an ANCOVA due to a lack of non-parametric alternative and the robustness of the F-test (Cochran, 1947;Glass, Peckham, & Sanders, 1972;Hack, 1958;Pearson, 1931). A Tukey's HSD post-hoc analysis reveals that Counterfactual was rated as more explainable than Probability Scores (p = 0.002), as shown in Figure A1 The reduced questionnaire, after a factor analysis and verification is given in Table A1.

No abstract