Digital Twins in Human-Computer Interaction: A Systematic Review

Fogli, Daniela

doi:10.1080/10447318.2022.2118189

Cited by 26 publications

(3 citation statements)

References 62 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The long-term formation of trust in automation and corresponding design patterns for monitoring and interventions are increasingly provided by the HCI research community [8,14], but their applicability across other tasks and contexts is not straightforward. Trust in automation systems may be even harder to accomplish when the underlying data is uncertain, such as when it is relying on learned data or digital twins [5]. In this context, ways to communicate the mechanisms defining their behavior and the origin of underlying data have to be presented in a trustable way [4,13].…”

Section: Monitor and Intervenementioning

confidence: 99%

Intervening, Teaming, Delegating: Creating Engaging Automation Experiences

Fröhlich

Baldauf

Palanque

et al. 2023

Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

Automated systems are becoming common in private, public and professional life. Given their increasing ubiquity and availability to a growing diversity of users, it is important to explore requirements, design principles, and user experience factors across application sectors and scientific disciplines. This workshop provides a forum for researchers and practitioners active in the field of "Automation Experience". In a keynote talk, a poster madness, discussions, and hands-on sessions, the participants will explore and discuss specific opportunities and challenges related to future forms of engagement with an increasing number of automated entities (automations). To this end, the focus topics for the workshop comprise (1) novel ways for monitoring of and intervening with increasingly intelligent agents and artifacts, (2) collaborative interaction to support teaming up and cooperation among humans and automations, and (3) orchestration and delegation of increasingly complex tasks to smart spaces. The results of the workshop are a set of research ideas and drafts of joint research initiatives to drive further automation experience research in a collaborative and interdisciplinary manner. CCS CONCEPTS• Human-centered computing → Human computer interaction (HCI).

show abstract

Section: Monitor and Intervenementioning

confidence: 99%

Intervening, Teaming, Delegating: Creating Engaging Automation Experiences

Fröhlich

Baldauf

Palanque

et al. 2023

Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

show abstract

“…However, in the literature on Digital Twins, home inhabitants usually play a passive role: they contribute to generating data but do not intervene in actively modifying smart home behaviors by creating routines. Moreover, there is a lack of scientific work addressing the issues related to the interaction between users and Digital Twins [31]. This paper explores the design and implementation of a Digital Twin for a green smart home to forecast and optimize the energy consumption of various appliances.…”

Section: Introductionmentioning

confidence: 99%

Enabling End-User Development in Smart Homes: A Machine Learning-Powered Digital Twin for Energy Efficient Management

Cotti,

Guizzardi,

Barricelli

et al. 2024

Future Internet

Self Cite

View full text Add to dashboard Cite

End-User Development has been proposed over the years to allow end users to control and manage their Internet of Things-based environments, such as smart homes. With End-User Development, end users are able to create trigger-action rules or routines to tailor the behavior of their smart homes. However, the scientific research proposed to date does not encompass methods that evaluate the suitability of user-created routines in terms of energy consumption. This paper proposes using Machine Learning to build a Digital Twin of a smart home that can predict the energy consumption of smart appliances. The Digital Twin will allow end users to simulate possible scenarios related to the creation of routines. Simulations will be used to assess the effects of the activation of appliances involved in the routines under creation and possibly modify them to save energy consumption according to the Digital Twin’s suggestions.

show abstract

“…referred to as ''emotion recognition in a digital twin''. The authors of a comprehensive study, [2], on human-computer interaction and digital twins provide recommendations for further research. Digital twins can be used in healthcare to monitor patients' emotional states for better care and in customer service to provide tailored services.…”

Section: Introductionmentioning

confidence: 99%

Emotion Recognition for Affective Human Digital Twin by Means of Virtual Reality Enabling Technologies

Amara¹,

Kerdjidj

Ramzan

2023

IEEE Access

View full text Add to dashboard Cite

Digital Twin is the seamless data integration between a physical and virtual machine in either direction. Emotion recognition in healthcare is becoming increasingly important due to recent developments in Machine Learning methods. However, it may face technical problems such as limited datasets, occlusion and lighting issues, identifying key features, incorrect emotion classification, high implementation costs, head posture, and a person's cultural background. This paper proposes a novel approach based on facial expression and body movement recognition for emotion recognition. It uses three devices (Kinect 1, Kinect 2, and RGB HD camera) to construct a new bi-modal database containing 17 participants' performances of six emotional states. Two mono-modal classifiers have been developed to obtain sufficient state information based on facial expression and body motion analysis. The system's performance is assessed using three algorithms: Bagged Trees, k-Nearest Neighbors (k-NN), and Support Vector Machine (Linear and Cubic). The acquired findings demonstrate the excellent performance of the suggested method and the effectiveness of the proposed features, particularly the combination of 3D distance and 3D angle, in characterising and identifying emotions. Results obtained using Kinect 2 marginally surpass those with Kinect 1. Comparing 2D RGB and RGB-D data reveals that the depth information significantly raises the recognition rate. RGB-D features can be used to represent emotions, but there are discrepancies between RGB and RG-D data.

show abstract

Digital Twins in Human-Computer Interaction: A Systematic Review

Cited by 26 publications

References 62 publications

Intervening, Teaming, Delegating: Creating Engaging Automation Experiences

Intervening, Teaming, Delegating: Creating Engaging Automation Experiences

Enabling End-User Development in Smart Homes: A Machine Learning-Powered Digital Twin for Energy Efficient Management

Emotion Recognition for Affective Human Digital Twin by Means of Virtual Reality Enabling Technologies

Contact Info

Product

Resources

About