Systematic Review of Multimodal Human–Computer Interaction

Azofeifa, Jose Daniel; Noguez, Julieta; Ruiz, Sergio; Molina-Espinosa, José Martín; Magana, Alejandra J.; Beneš, Bedřich

doi:10.3390/informatics9010013

Cited by 22 publications

(9 citation statements)

References 127 publications

(134 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Conversely, the visualtactile congruent stimulation slightly increased the Kappa illusion in the BC condition with respect to the unimodal conditions. In this regard, the illusory effects found in this study in the visual and the tactile domains are expected to increase in VR, as the perceptual priors experienced in the real world can be strongly biased concerning both the modalities [8]; moreover, the visual-tactile crossmodal integration could improve the sense of immersion and the customization of the experience [36].…”

Section: Discussionmentioning

confidence: 85%

Toward the manipulation of time and space in extended reality: a preliminary study on multimodal Tau and Kappa illusions in the visual-tactile domain

Pra

Catrambone

Wassenhove

et al. 2022

2022 31st IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)

View full text Add to dashboard Cite

In the last few years, Extended reality (XR) has enabled novel forms of sensory experiences and social interplay, which can be hardly experienced in real life. However, the full potential of XR has not been exploited yet, since vision remains the main interaction modality, and the time-and spacemodulation of the sense of self -which could open interesting perspectives in several scenarios -is still largely unexplored. To pave the path to a multi-modal manipulation of the sense of time and space in immersive XR, in this work we discuss the preliminary outcomes of the first investigation in the visualtactile domain of two well known perceptual illusions affecting spatial and temporal perception, i.e. Tau and Kappa effects, respectively. We compared the effects originated from unimodal stimulation (i.e., only visual or tactile) with the same effects induced by convergent bimodal stimulation (i.e., visual and tactile), delivered to the forearm. Results show that both Tau and Kappa effects are affected by the multi-modality of the stimulation, and that the perceptual bias differently affects time-or space-perception based on the modality used for stimulus delivery. Our results, although preliminary, seem to suggest that multimodal perceptual illusions could be a viable solution for time-and space-modulation of the sense of self in immersive XR and advanced social human-robot interaction.

show abstract

Section: Discussionmentioning

confidence: 85%

Toward the manipulation of time and space in extended reality: a preliminary study on multimodal Tau and Kappa illusions in the visual-tactile domain

Pra

Catrambone

Wassenhove

et al. 2022

2022 31st IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)

View full text Add to dashboard Cite

show abstract

“…Hassenzahl & Tractinsky (2006) stressed the lack of empirical studies in the specific field of Human Computer Interaction (HCI). Only recently, thanks to technological development, most studies focused their attention on HCI (Azofeifa et al 2022). UX has been investigated and applied to develop new information and communications technology (ICT) devices and other emerging interaction technologies such as virtual reality (VR) or augmented reality (AR).…”

Section: Introductionmentioning

confidence: 99%

Fundamentals and issues of user experience in the process of designing consumer products

Berni¹,

Borgianni²,

Basso³

et al. 2023

Des. Sci.

View full text Add to dashboard Cite

User experience (UX) application in the practice of engineering and product design is still limited. The present paper provides insights into research on UX design and recommendations for design practitioners by pointing out common criticalities. These outcomes are achieved through a literature review on how UX relates to design. First, issues in benefitting from UX understanding in design are identified with a specific focus on theoretical contributions. Second, experimental papers investigating UX and design are analysed in relation to previously identified issues. Although issues are present to some extent in all the contributions, the empirical studies dealing with UX in design are overall valid. The results highlight UX’s support in revealing design requirements, but its capability of steering design processes is arguable, as concrete guidelines for practitioners are not well described. Based on identified issues, the authors propose a checklist to make UX studies in design more reliable and their outcomes more comparable.

show abstract

“…However, it can also act as a cache for multimedia content, thus allowing devices to retrieve that content locally, thus reducing latency [27]. Multimodal interfaces offer highly efficient and expressive human-computer interaction [28]. Therefore, this study is aimed at establishing a theoretical basis for the use of multimodal teaching mode in teaching Chinese as a foreign language by using multiple modalities to convey information and sensory perception.…”

Section: Introductionmentioning

confidence: 99%

Application of Multimodal Multimedia Information and Big Data Technology in Teaching Chinese as a Foreign Language Course

Nie

2023

International Journal of Digital Multimedia Broadcasting

View full text Add to dashboard Cite

With the change of the times, under the leadership of big data, cloud computing, network technology, mobile Internet, and other high technology, education has gradually broken free from the traditional teaching mode. At the same time, advanced technology has also lifted the confinement of traditional teaching mode in space and time and opened the skylight of intelligent thinking mode. From the current situation of Chinese language fever around the world, the demand of Chinese language learners for learning materials and learning resources will continue to increase in the future. With the continuous development of Internet education, the combination of Internet technology and international Chinese education will definitely become the key direction for the development of international Chinese education. In this context, online teaching resources will become an important basis for the development of “Internet+Chinese international education”, so it is necessary to investigate and study the online teaching resources of “Internet+Chinese international education.” As a matter of fact, with the development of society and the advancement of technology, the era of informationization has come. As a result, the main theme of education in the information age is to provide suitable education for students who want to learn Chinese and to promote the active and lively development of each student. In the past, the one-size-fits-all education model of classroom teaching could not meet the individual development of students and the demand of society for diversified talents. Therefore, the traditional teaching of Chinese as a foreign language is in urgent need of change, and personalized teaching is attracting attention. With the emergence of technologies such as cloud computing, Internet of Things, and big data in education, personalized teaching has received technical support. This paper is aimed at exploring how to apply the new technologies in the teaching process to help teachers personalize teaching, stimulate students’ interest in learning, meet their individual needs, break through the traditional teaching methods, and make it possible to teach according to their abilities. As one of the main learning resources, the quality of multimedia courseware will be the criterion to measure whether it meets the teaching and learning needs. Consequently, the use of big data, multimedia, and multimodal technologies to improve the Chinese language material library and develop special software for teaching Chinese as a foreign language, as well as the production of high-quality multimedia courseware that is highly compatible with the teaching materials, will become the trend of future research.

show abstract

Systematic Review of Multimodal Human–Computer Interaction

Cited by 22 publications

References 127 publications

Toward the manipulation of time and space in extended reality: a preliminary study on multimodal Tau and Kappa illusions in the visual-tactile domain

Toward the manipulation of time and space in extended reality: a preliminary study on multimodal Tau and Kappa illusions in the visual-tactile domain

Fundamentals and issues of user experience in the process of designing consumer products

Application of Multimodal Multimedia Information and Big Data Technology in Teaching Chinese as a Foreign Language Course

Contact Info

Product

Resources

About