Algorithmic Long-Term Unemployment Risk Assessment in Use: Counselors’ Perceptions and Use Practices

Abstract

The recent surge of interest in algorithmic decision-making among scholars across disciplines is associated with its potential to resolve the challenges common to administrative decision-making in the public sector, such as greater fairness and equal treatment of each individual, among others. However, algorithmic decision-making combined with human judgment may introduce new complexities with unclear consequences. This article offers evidence that contributes to the ongoing discussion about algorithmic decision-making and governance, contextualizing it within a public employment service. In particular, we discuss the use of a decision support system that employs an algorithm to assess individual risk of becoming long-term unemployed and that informs counselors to assign interventions accordingly. We study the human interaction with algorithms in this context using the lenses of human detachment from and attachment to decision-making. Employing a mixed-method research approach, we show the complexity of enacting the potentials of the data-driven decision-making in the context of a public agency.

Think About the Stakeholders First! Towards an Algorithmic Transparency Playbook for Regulatory Compliance

“…Overall, researchers found that the explanations improved the confidence of the decisions, but counter-intuitively, had a somewhat negative effect on the quality of those decisions [80].…”

“…Running-example. In the IEFP use case, SHAP factors were given to job counselors to show the top factors influencing the score of a candidate both positively and negatively [80]. The transparency provided by SHAP provided a local explanation about the outcome of the model.…”

“…Having a HITL is an increasingly common practice for implementing ADS [24,56,77]. The ADS assigns unemployed persons as low, medium, or high risk for remaining unemployed, and then job counselors have the responsibility of assigning them to interventions such as re-skilling, resume building, or job search training [80].…”

The Adoption of Artificial Intelligence in Health Care and Social Services in Australia: Findings From a Methodologically Innovative National Survey of Values and Attitudes (the AVA-AI Study)

“…However, there is a small body of research documenting service users' experiences of specific AI applications in the social services, particularly users' negative experiences of exclusion and discrimination [21,33], providing context-specific insights into system users' experiences of AI and illustrating the high-stakes nature of implementing AI in this domain. This work, together with some small-scale, mostly qualitative studies involving frontline social service staff [34][35][36][37][38], illustrates the complex and dynamic relationship between AI and the routines of social welfare professionals and indicates mixed reactions to these systems among staff. For example, the study by Zejnilović et al [36] of counselors in a Portuguese employment service in 2020 found high levels of distrust and generally negative perceptions of an AI system used to score clients' risk of long-term unemployment.…”

Mind the gap! On the future of AI research

“…The system essentially categorised the unemployed as good or bad investments, leading the Human Rights Commissioner to establish the algorithm's decisions as unjust, and in the end the system was banned (Kuziemski and Misuraca, 2020). A central issue here was that there was no clear plan for how human judgement and algorithmic decision-making could be joined and enacted in a public service context (Zejnilovic et al, 2020).…”