Understanding perception of algorithmic decisions: Fairness, trust, and emotion in response to algorithmic management

Abstract

Algorithms increasingly make managerial decisions that people used to make. Perceptions of algorithms, regardless of the algorithms' actual performance, can significantly influence their adoption, yet we do not fully understand how people perceive decisions made by algorithms as compared with decisions made by humans. To explore perceptions of algorithmic management, we conducted an online experiment using four managerial decisions that required either mechanical or human skills. We manipulated the decision-maker (algorithmic or human), and measured perceived fairness, trust, and emotional response. With the mechanical tasks, algorithmic and human-made decisions were perceived as equally fair and trustworthy and evoked similar emotions; however, human managers' fairness and trustworthiness were attributed to the manager's authority, whereas algorithms' fairness and trustworthiness were attributed to their perceived efficiency and objectivity. Human decisions evoked some positive emotion due to the possibility of social recognition, whereas algorithmic decisions generated a more mixed response-algorithms were seen as helpful tools but also possible tracking mechanisms. With the human tasks, algorithmic decisions were perceived as less fair and trustworthy and evoked more negative emotion than human decisions. Algorithms' perceived lack of intuition and subjective judgment capabilities contributed to the lower fairness and trustworthiness judgments. Positive emotion from human decisions was attributed to social recognition, while negative emotion from algorithmic decisions was attributed to the dehumanizing experience of being evaluated by machines. This work reveals people's lay concepts of algorithmic versus human decisions in a management context and suggests that task characteristics matter in understanding people's experiences with algorithmic technologies.

Highly automated job interviews: Acceptance under the influence of stakes

“…On the other hand, this is in line with former research that suggested that feelings of uncertainty and ambiguity are especially prevalent in novel situations where people do not know what to do, what to feel, or how to judge the situations (Langer & König, ; Tene & Polonetsky, ). This might also account for the result that the low‐stake interview was perceived as being less ambiguous, as it might be more familiar to use automation for low‐stake scenarios such as training or work scheduling (see also Lee, ; Zyda, ). However, we were surprised by the fact that there was no difference in emotional creepiness between the groups, not even for automated high‐stake interviews.…”

“…Both studies suggest that people get less credit for their influence on the decision‐making process as soon as there are decision‐support systems involved, which also implies that people believe that human influence on decisions reduces with automatic decision‐support. In a similar vein, Lee () found that people react negatively to automatic decisions in personnel selection situations compared to automatic decisions in work scheduling situations and argued that this difference is due to the reduction of human influence on the decision. On the positive side, reducing the human influence on decisions might also be perceived to reduce possible biases and thus enhance consistency in the decision‐making process (which people seem to tend to believe when thinking about automatic decisions; Miller, Katz, & Gans, ).…”

“…Applying highly automated tools within different contexts could affect people's reactions to these tools. Lee () found that people react more favorably to highly automated decisions for mechanical tasks (e.g., work scheduling) than for human tasks (e.g., hiring), which might be explained by what was at stakes. Decisions about work scheduling might affect if a person can go to the cinema or meet with friends.…”

“…We propose that the stakes of a context might affect reactions to highly automated tools because people might perceive that a human decision agent should be in charge of high‐stake processes and decisions instead of leaving those to highly automated tools (Binns et al, ). Additionally, setting up a high‐stake context in a virtual environment might also lead to negative feelings about the respective tool as people conclude that this kind of situation should rather be handled through an interaction with another human being maybe because of the implicit sign that there is more social recognition when another person handles an important decision (Lee, ). To examine whether stakes might influence people's reactions to highly automated tools, we manipulated the stakes through introducing participants to either a personnel selection or a training situation.…”

“…We then examined participants’ reactions to such a tool in comparison to a classical technology‐mediated interview (i.e., a videoconference interview). Furthermore, we wanted to advance prior research (Lee, ) that indicated that the context matters regarding people's reactions to automated tools. We argue that what is at stakes during a situation (i.e., the relevance of the situation for the future of the person; Peeters & Lievens, ) might be an explanation for why highly automated processes are accepted within one situation compared to another.…”