Social Virtual Agents and Loneliness: Impact of Virtual Agent Anthropomorphism on Users’ Feedbacks

Zehnder, Eloïse; Dinet, Jérôme; Charpillet, François

doi:10.1007/978-3-030-80091-8_33

Cited by 3 publications

(2 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The risk of anthropomorphism in AI systems is widely discussed -with concerns that humans may too readily befriend or empathise with anthropomorphised agents [197,189], leading to privacy risks in encouraging the sharing of intimate information [32,242]. In a study, Kronemann et al [122] find that personalisation positively influenced consumer intentions to disclose personal information to a digital assistant.…”

Section: Ir5 Anthropomorphismmentioning

confidence: 99%

Personalisation within bounds: A risk taxonomy and policy framework for the alignment of large language models with personalised feedback

Kirk¹,

Vidgen²,

Röttger³

et al. 2023

Preprint

View full text Add to dashboard Cite

Large language models (LLMs) are used to generate content for an increasingly wide range of tasks, and are set to reach a growing audience in coming years due to integration in product interfaces like ChatGPT or search engines like Bing. This intensifies the need to ensure that models are aligned with human preferences and do not produce unsafe, inaccurate or toxic outputs. While alignment techniques like reinforcement learning with human feedback (RLHF) and red-teaming can mitigate some safety concerns and improve model capabilities, it is unlikely that an aggregate fine-tuning process can adequately represent the full range of users' preferences and values. Different people may legitimately disagree on their preferences for language and conversational norms, as well as on values or ideologies which guide their communication. Personalising LLMs through micro-level preference learning processes may result in models that are better aligned with each user. However, there are several normative challenges in defining the bounds of a societally-acceptable and safe degree of personalisation. In this paper, we ask how, and in what ways, LLMs should be personalised. First, we review literature on current paradigms for aligning LLMs with human feedback, and identify issues including (i) a lack of clarity regarding what alignment means; (ii) a tendency of technology providers to prescribe definitions of inherently subjective preferences and values; and (iii) a "tyranny of the crowdworker", exacerbated by a lack of documentation in who we are really aligning to. Second, we present a taxonomy of benefits and risks associated with personalised LLMs, for individuals and society at large. Finally, we propose a three-tiered policy framework that allows users to experience the benefits of personalised alignment, while restraining unsafe and undesirable LLM-behaviours within (supra-)national and organisational bounds.

show abstract

Section: Ir5 Anthropomorphismmentioning

confidence: 99%

Personalisation within bounds: A risk taxonomy and policy framework for the alignment of large language models with personalised feedback

Kirk¹,

Vidgen²,

Röttger³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…There are some studies focusing on Replika, which mostly focused on its function as a companion, marketed as 'always here to listen and talk' (Luka Inc., 2022). Ta et al (2020) analyzed online user reviews and found that users stated that Replika can ameliorate loneliness and offer everyday social support, and Zehnder et al (2021) explored how the evolution of Replika's appearance impacts perceived anthropomorphism expressed on Internet user reviews. Xie and Pentina (2022) analyzed Replika user interviews and concluded that, under conditions of stress and lack of human companionship, individuals can develop strong bonds with their bots, if they perceive the bots as providing them with emotional support.…”

Section: Ai and Gender Imaginariesmentioning

confidence: 99%

Ideal technologies, ideal women: AI and gender imaginaries in Redditors’ discussions on the Replika bot girlfriend

Depounti

Saukko

Natale

2022

Media, Culture & Society

View full text Add to dashboard Cite

There is extensive literature on how expectations and imaginaries about artificial intelligence (AI) guide media and policy discussions. However, it has not been considered how such imaginaries are activated when users interact with AI technologies. We present findings of a study on how users on a subreddit discussed ‘training’ their Replika bot girlfriend. The discussions featured two discursive themes that focused on the AI imaginary of ideal technology and the gendered imaginary of the ideal bot girlfriend. Users expected their AI Replikas to both be customizable to serve their needs and to have a human-like or sassy mind of their own and not spit out machine-like answers. Users thus projected dominant notions of male control over technology and women, mixed with AI and postfeminist fantasies of ostensible independence onto the interactional agents and activated similar scripts embedded in the devices. The vicious feedback loop consolidated dominant scripts on gender and technology whilst appearing novel and created by users. While most research on the use of AI is conducted in applied computer science to improve user experience, this article outlines a media and cultural studies lens for a critical understanding of these emerging technologies as they become embedded in communication and meaning-making.

show abstract

The Impact of Artificial Intelligence on Human Sexuality: A Five-Year Literature Review 2020–2024

Döring,

Le,

Vowels

et al. 2024

Curr Sex Health Rep

View full text Add to dashboard Cite

Purpose of Review Millions of people now use generative artificial intelligence (GenAI) tools in their daily lives for a variety of purposes, including sexual ones. This narrative literature review provides the first scoping overview of current research on generative AI use in the context of sexual health and behaviors. Recent Findings The review includes 88 peer-reviewed English language publications from 2020 to 2024 that report on 106 studies and address four main areas of AI use in sexual health and behaviors among the general population: (1) People use AI tools such as ChatGPT to obtain sexual information and education. We identified k = 14 publications that evaluated the quality of AI-generated sexual health information. They found high accuracy and completeness. (2) People use AI tools such as ChatGPT and dedicated counseling/therapy chatbots to solve their sexual and relationship problems. We identified k = 16 publications providing empirical results on therapists’ and clients’ perspectives and AI tools’ therapeutic capabilities with mixed but overall promising results. (3) People use AI tools such as companion and adult chatbots (e.g., Replika) to experience sexual and romantic intimacy. We identified k = 22 publications in this area that confirm sexual and romantic gratifications of AI conversational agents, but also point to risks such as emotional dependence. (4) People use image- and video-generating AI tools to produce pornography with different sexual and non-sexual motivations. We found k = 36 studies on AI pornography that primarily address the production, uses, and consequences of – as well as the countermeasures against – non-consensual deepfake pornography. This sort of content predominantly victimizes women and girls whose faces are swapped into pornographic material and circulated without their consent. Research on ethical AI pornography is largely missing. Summary Generative AI tools present new risks and opportunities for human sexuality and sexual health. More research is needed to better understand the intersection of GenAI and sexuality in order to a) help people navigate their sexual GenAI experiences, b) guide sex educators, counselors, and therapists on how to address and incorporate AI tools into their professional work, c) advise AI developers on how to design tools that avoid harm, d) enlighten policymakers on how to regulate AI for the sake of sexual health, and e) inform journalists and knowledge workers on how to report about AI and sexuality in an evidence-based manner.

show abstract

Social Virtual Agents and Loneliness: Impact of Virtual Agent Anthropomorphism on Users’ Feedbacks

Cited by 3 publications

References 17 publications

Personalisation within bounds: A risk taxonomy and policy framework for the alignment of large language models with personalised feedback

Personalisation within bounds: A risk taxonomy and policy framework for the alignment of large language models with personalised feedback

Ideal technologies, ideal women: AI and gender imaginaries in Redditors’ discussions on the Replika bot girlfriend

The Impact of Artificial Intelligence on Human Sexuality: A Five-Year Literature Review 2020–2024

Contact Info

Product

Resources

About