Malicious Design in AIVR, Falsehood and Cybersecurity-oriented Immersive Defenses

Abstract

Advancements in the AI field unfold tremendous opportunities for society. Simultaneously, it becomes increasingly important to address emerging ramifications. Thereby, the focus is often set on ethical and safe design forestalling unintentional failures. However, cybersecurity-oriented approaches to AI safety additionally consider instantiations of intentional malice -including unethical malevolent AI design. Recently, an analogous emphasis on malicious actors has been expressed regarding security and safety for virtual reality (VR). In this vein, while the intersection of AI and VR (AIVR) offers a wide array of beneficial cross-fertilization possibilities, it is responsible to anticipate future malicious AIVR design from the onset on given the potential socio-psycho-technological impacts. For a simplified illustration, this paper analyzes the conceivable use case of Generative AI (here deepfake techniques) utilized for disinformation in immersive journalism. In our view, defenses against such future AIVR safety risks related to falsehood in immersive settings should be transdisciplinarily conceived from an immersive co-creation stance. As a first step, we motivate a cybersecurity-oriented procedure to generate defenses via immersive design fictions. Overall, there may be no panacea but updatable transdisciplinary tools including AIVR itself could be used to incrementally defend against malicious actors in AIVR.

Transdisciplinary AI Observatory—Retrospective Analyses and Future-Oriented Contradistinctions

“…The aim of the research cluster is to demonstrate the feasibility of malicious AI design motivated by diverse adversarial goals across a variety of domains in order to foster safety-awareness. Beyond that, we consider 1 extra emerging risk pattern, namely automated disconcertion [28] which we introduce in a few paragraphs.…”

“…Interestingly, an already perceptible consequence of the mere existence of risk Ia instantiations containing the design of deepfake technologies already led to the emergence of a risk pattern which has been termed automated disconcertion [28]. Automated disconcertion can imply the intentional or also unintentional mislabelling of real samples as fake-for example, in the context of misleading conspiracy theories [50] or against the background of uncertain political settings as it was the case in Gabon not long ago [51].…”

“…Automated disconcertion can imply the intentional or also unintentional mislabelling of real samples as fake-for example, in the context of misleading conspiracy theories [50] or against the background of uncertain political settings as it was the case in Gabon not long ago [51]. (To summarize the latter, a "recent failed military coup in the context of pre-existing political unrest in Gabon was partially grounded in the proliferation of the wrong assumption that an official presidential video represented a manipulative deepfake video" [28]). Conversely, automated disconcertion can also mean that fake samples are considered as being authentic or simply lead to highly uncertain and inconclusive settings in which doubts cannot be further resolved in reasonable time with acceptable resources.…”

“…Recently, co-creation design fictions (DFs) [117,118] known from human-computer interaction (HCI) [119] have been recommended for security practices in the AI field [120] and at the intersection between AI and virtual reality (AIVR) [28]. Generally, DFs "can be used for technological future projections by experts in the form of for example, narratives or construed prototypes that can be represented in text, audio or video formats but also in VR environments" [28] (whereby the authors cautioned likewise not to regard a DF as a means to predict the future but as preparatory tool). In our view, one promising way to perform an RDA-based RCRA could be to frame each RCRA cluster as co-creation DF task.…”

Does Mixed Reality Have a Cassandra Complex?

“…For example, a deepfake hologram portraying a movie celebrity sharing political propaganda which the celebrity themselves don't endorse, targeting fans and spreading lies about the incumbent leader's political opponents. The hologram could be made to harass or provoke viewers (Aliman and Kester, 2020), goading them into acting irrationally. This warrants ethical considerations when designing XR experiences for broadcasting and entertainment; 2) XR technology which can sense and interpret objects in the environment can be used to mask and/or delete recognized objects.…”