Missing Mechanisms of Manipulation in the EU AI Act

Abstract

The European Union Artificial Intelligence (AI) Act proposes to ban AI systems that ”manipulate persons through subliminal techniques or exploit the fragility of vulnerable individuals, and could potentially harm the manipulated individual or third person”. This article takes the perspective of cognitive psychology to analyze and understand what algorithmic manipulation consists of, who vulnerable individuals may be, and what is considered as harm. Subliminal techniques are expanded with concepts from behavioral science and the study of preference change. Individual psychometric differences which can be exploited are used to expand the concept of vulnerable individuals. The concept of harm is explored beyond physical and psychological harm to consider harm to one's time and right to an un-manipulated opinion. The paper offers policy recommendations that extend from the paper's analyses.

What Is a Subliminal Technique? An Ethical Perspective on AI-Driven Influence

“…However, despite the long tradition of research on the potential to influence behavior through subliminal perception, much disagreement remains about the phenomenon's scope and practical relevance. Researchers agree that the effects of subliminal stimuli on behavior, if they exist, are much weaker than popular discussions assume [11]. They are extremely short-lived, tending to leave no traces of lasting effects beyond 100ms ( [15], [16]), and seem capable of triggering actions only if the individual already intends to perform them [17].…”

“…While these behaviors are often portrayed as proxies for the satisfaction of user preferences, there is concern in among media and the public that they can become misaligned in ways that instead exploit and reinforce negative emotions, such as outrage or negative social comparison [30]. Existing studies suggest that automated recommendations can change user preferences instead of learning how to satisfy them ( [11], [31], [32]).…”

“…Exactly what kinds of technologies might be covered by article 5(1)(a) remains uncertain. Part of this uncertainty relates to the term 'subliminal techniques' ( [6]- [11]). The term does not have an established meaning in prior EU law, nor is it defined in the AIA.…”

“…So far, most commentary on this issue has focused on how the legal text itself might be changed to address the issue of scope, for example by removing the term 'subliminal techniques' or supplementing it with a more precise definition ( [6], [10], [11]). In this paper we pursue a different strategy.…”

Dark Patterns, Enforcement, and the emerging Digital Design Acquis: Manipulation beneath the Interface

“…However, important harms need to be considered, such as societal harms "harming the democratic process, eroding the rule of law, or exacerbating inequality", 185 time, addiction and autonomy. 186 Article 5(1)(b) of the AI Act prohibits AI practices involving systems that exploit and target the vulnerabilities of specific groups of individuals, such as age, disability, or socio-economic status, to manipulate behaviour. However, this provision only regards "specific groups".…”