Adapting the System to Users Based on Implicit Data: Ethical Risks and Possible Solutions

Abstract

Abstract. Symbiotic systems are systems that gather personal data implicitly provided by the user, derive a profile/model of the user from such data and adjust their output/service according to their notion of what would be desirable to the user thus modeled. Because of these three characteristics, symbiotic systems represent a step forward towards facilitated, simplified, user-friendly digital devices, or do they? Here we propose three cases describing realistic applications of symbiotic systems that potentially encapsulate some serious risk to their users. Experts of five different domains (i.e., ethics, security, law, human-computer interaction and psychology) dissect each case to identify the risks to the users and derive some possible minimization strategies. This panel aims at contributing to a beneficial development of symbiotic systems as it can be achieved by increasing users' discernment and awareness of their consequences for society and everyday life.

Assessment of Implicit and Explicit Measures of Mental Workload in Working Situations: Implications for Industry 4.0

“…Regarding the second limit, namely the system transparency, it is relevant to clarify to the users the reasons and the modalities of the machine's actions that modify the "state" of the world. This characteristic allows the users to carefully decide which data to provide to the devices and how to use the information coming from the devices [14]. Whenever this transparency is lacking, the system becomes a black box, and the user is not capable of understanding the causes of modifications in the system's functioning.…”

Conceptualization of the Human-Machine Symbiosis – A Literature Review

“…A third main objective of HMS is to create humanlike communication and interaction, which is crucial to enable actors to accept each other as equal partners [39]. Humanlike communication includes both verbal (spoken and written language) and non-verbal communication (gestures, facial expressions, emotions) [40,44]. The machine has to understand the different aspects of human communication and use them in an exchange with the human.…”

“…In addition to awareness of the environment, the machine must also have an understanding of the actors and the system [39]. This requires that the machine possess a model or profile of the human, the machine itself, and the system in which data can be collected [23,44]. This will allow the machine to respond optimally to all interactions, and to gain new insights [11,31].…”

“…This will allow the machine to respond optimally to all interactions, and to gain new insights [11,31]. This representation, as the literature emphasizes in particular, can be successful only if there is constant transparency [23,44], that is, if the machine discloses what data it collects and possesses and humans are not allowed to withhold any information [23]. In this context, security is a decisive factor [38].…”

“…The task model, meanwhile, contains all information on the tasks and their process status (c14) [11,25]. HMS must, therefore, be designed in such a way that information is available or can be obtained at any time by the actors (r2) [44,45]. The section above on requirements already elaborated that the machine must store the data needed for this purpose (r3) [23,44].…”