Sensitivity to interventions and the relationship with numeracy

Abstract

The main goal of this research was to investigate whether people exhibit algorithm aversion—a tendency to avoid using an imperfect algorithm even if it outperforms human judgments—in the case of estimating students’ percentile scores on a standardized math test. We also explored the relationships between numeracy and algorithm aversion and tested two interventions aimed at reducing algorithm aversion. In two studies, we asked participants to estimate the percentiles of 46 real 15-year-old Polish students on a standardized math test. Participants were offered the opportunity to compare their estimates with the forecasts of an algorithm—a statistical model that predicted real percentile scores based on fi ve explanatory variables (i.e., gender, repeating a class, the number of pages read before the exam, the frequency of playing online games, socioeconomic status). Across two studies, we demonstrated that even though the predictions of the statistical model were closer to students’ percentile scores, participants were less likely to rely on the statistical model predictions in making forecasts. We also found that higher statistical numeracy was related to a higher reluctance to use the algorithm. In Study 2, we introduced two interventions to reduce algorithm aversion. Depending on the experimental condition, participants either received feedback on statistical model predictions or were provided with a detailed description of the statistical model. We found that people, especially those with higher statistical numeracy, avoided using the imperfect algorithm even though it outperformed human judgments. Interestingly, a simple intervention that explained how the statistical model works led to better performance in an estimation task

M.K. Gandhi – “An Autobiography or Story of my Experiments with Truth”

“…(Gandhi,280) This argument corresponds to what is known as the phenomenon of algorithm aversion, which was demonstrated in a seminal experiment by Dietvorst, Simmons, and Massey (2015). (See also Dzieżyk, Hetmańczuk, Traczyk, 2020. ) These authors constructed a situation in which students could experience imperfect forecasts made either by an algorithm or by humans.…”