Can You Ink While You Blink? Assessing Mental Effort in a Sensor-Based Calligraphy Trainer

Abstract

Sensors can monitor physical attributes and record multimodal data in order to provide feedback. The application calligraphy trainer, exploits these affordances in the context of handwriting learning. It records the expert’s handwriting performance to compute an expert model. The application then uses the expert model to provide guidance and feedback to the learners. However, new learners can be overwhelmed by the feedback as handwriting learning is a tedious task. This paper presents the pilot study done with the calligraphy trainer to evaluate the mental effort induced by various types of feedback provided by the application. Ten participants, five in the control group and five in the treatment group, who were Ph.D. students in the technology-enhanced learning domain, took part in the study. The participants used the application to learn three characters from the Devanagari script. The results show higher mental effort in the treatment group when all types of feedback are provided simultaneously. The mental efforts for individual feedback were similar to the control group. In conclusion, the feedback provided by the calligraphy trainer does not impose high mental effort and, therefore, the design considerations of the calligraphy trainer can be insightful for multimodal feedback designers.

A New Era in Multimodal Learning Analytics: Twelve Core Commitments to Ground and Grow MMLA

“…It is not surprising, then, that this same principle should carry over into how feedback or multimodal data should be shared with researchers and participants. Development of multimodal feedback has been prevalent within the HCI community (Ciordas-Hertel, 2020;Freeman et al, 2017;Limbu, Jarodzka, Klemke, & Specht, 2019) but has scarcely been explored within the MMLA community (Worsley & Ochoa, 2020). Instead, there has been a tendency to forget about multimodality as soon as the data have been collected and analyzed, resorting to traditional charts and figures in a dynamic dashboard in order to display data.…”

Table Tennis Tutor: Forehand Strokes Classification Based on Multimodal Data and Neural Networks

“…Moreover, these sensors are typically synchronised with human activities so that one can analyse historical evidence of learning activities and the description of the learning process [23]. Limbu et al [6] developed the "Calligraphy Tutor", which uses Pen sensors in Microsoft Surface and EMG sensors in Myo (Myo sensor armbandhttps://www.rs-online.com/designspark/expanding-gesture-control-with-the-myo, last accessed on 25 March 2021) sensor armband to provide feedback to learners during practice. It also allows the calligraphy teacher to create an expert model, which the learners can later use to practice and receive guidance and feedback based on the expert model.…”

“…Moreover, the beginners could also develop improper techniques that, once automated, are difficult to change [ 5 ]. Limbu et al [ 6 ], in their work, have pursued to address this issue by using sensors to record expert models and use the expert model to train beginners. In line with their work, this study aims to address this issue of shortage of mentors for training in table tennis by using sensors to complement mentors in providing immediate real-time feedback to the beginners.…”

Advanced Sensors Technology in Education

“…Limbu et al presents a pilot study done with the calligraphy trainer to evaluate the mental effort required by various types of feedback provided by the application [7]. The participants use the application to learn three characters from the Devanagari script.…”