Multisensory Perception and Learning: Linking Pedagogy, Psychophysics, and Human–Computer Interaction

Abstract

In this review, we discuss how specific sensory channels can mediate the learning of properties of the environment. In recent years, schools have increasingly been using multisensory technology for teaching. However, it still needs to be sufficiently grounded in neuroscientific and pedagogical evidence. Researchers have recently renewed understanding around the role of communication between sensory modalities during development. In the current review, we outline four principles that will aid technological development based on theoretical models of multisensory development and embodiment to foster in-depth, perceptual, and conceptual learning of mathematics. We also discuss how a multidisciplinary approach offers a unique contribution to development of new practical solutions for learning in school. Scientists, engineers, and pedagogical experts offer their interdisciplinary points of view on this topic. At the end of the review, we present our results, showing that one can use multiple sensory inputs and sensorimotor associations in multisensory technology to improve the discrimination of angles, but also possibly for educational purposes. Finally, we present an application, the ‘RobotAngle’ developed for primary (i.e., elementary) school children, which uses sounds and body movements to learn about angles.

Multisensory stimulation and its effect on breast milk volume production in mothers of premature infants

“…The simultaneous use of the senses creates a stimulating experience that effectively captures the recipient's attention and produces more intense emotions. As a result, information retention is enhanced ( 41 ) and a lasting experience is generated, creating an emotional connection with the recipient ( 31 ). Furthermore, a study by Rolls et al ( 38 ), reported a link between visual, olfactory and auditory stimuli and the learning process, as both the amygdala and the hippocampus interact with the emotion produced by the stimulus, generating learning that is demonstrated in social and emotional behavior ( 42 ).…”

Design and Optimization of Human-Computer Interaction System for Education Management Based on Artificial Intelligence

“…The intelligent control model, based on gesture and speech interaction, has effectively promoted the rapid development and updating of human-computer interaction in education management. It also provides a more efficient means for educational management [6][7].…”