“I’m Disabled and Married to a Foreign Single Mother”. Public Service Chatbot’s Advice on Citizens’ Complex Lives

Abstract

This paper describes a study of citizens' chats with a chatbot of a public agency. We have analyzed chat logs and identified citizens' lack of domain knowledge as a source of inadequate or failed chatbot responses. We identify three types of lack of domain knowledge: lack of the right vocabulary, uncertainty if a regulation fits the citizen's situation or the "shape sorting box" problem, or citizen's misunderstanding the regulations. The most serious failure is when a misunderstanding is not detected and corrected during the chat. The chatbot we studied is not able to make sense of badly formed questions from citizens. As implications for design we suggest making the chatbot limitations visible by not presenting it as a human-like avatar with a name. We also suggest to enable domain knowledge learning through its conversations.

When the Social Becomes Non-Human: Young People's Perception of Social Support in Chatbots

“…This push, with little attention to human needs and experiences, can lead to unintended negative consequences, such as online addiction and loneliness [20]. Chatbots may also present risks, such as biases, inadequate and failed responses [21], and privacy issues [6,22], all of which can negatively afect the quality of the experience of chatbots as a source of social support.…”

Innbyggernes møte med en digitalisert offentlig forvaltning

“…Vår forskning omfatter også studier av bruken av Navs chatbot Frida fra den ble lansert i 2018 og til i dag (Simonsen et al, 2020;Verne et al, 2022b). Denne studien omfatter omkring 10 000 logger av reelle chatbot-samtaler med innbyggere, i tillegg til intervjuer med telefon-og chat-veiledere samt chatbot-trenere (Steinstø, 2020;Simonsen, 2019;Løkken, 2023).…”

“…Frida svarer ikke på spørsmål om individuelle innbyggeres saker, den brukes anonymt. Den svarer greit på enkle spørsmål som angår et av Navs ytelsesområder ved å henvise til den regelen det er sannsynlig at innbyggeren spør om (Simonsen et al, 2020). Svarene er derfor alltid korrekte, men det er ikke alltid de er utfyllende svar på det innbyggeren spør om.…”

“Please talk to me! Or not?” Eine Untersuchung der Zufriedenheit mit Chatbots in der öffentlichen Verwaltung

“…Chatbots sollen mit Rechtschreibfehlern und Beleidigungen umgehen können (Espig et al, 2019;Simonsen et al, 2020).…”

“…Verschiedene Studien zeigen, dass die Zufriedenheit mit Chatbots eng damit zusammenhängt, ob diese korrekte und verlässliche Antworten liefern (Aoki, 2020; Kohne et al, 2020). Kann ein Chatbot eine Anfrage nicht verstehen, sind Hilfestellungen zu liefern, damit Nutzerinnen und Nutzer ihre Anfrage neu formulieren können (Simonsen et al, 2020). Limitationen des Chatbots sollen ausdrücklich als solche gekennzeichnet werden (Simonsen et al, 2020).…”