Read Between the Lines

Mitri, Daniele Di; Schneider, Ján; Klemke, Roland; Specht, Marcus; Drachsler, Hendrik

doi:10.1145/3303772.3303776

Cited by 35 publications

(26 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The technological approach used for the experiment is the MMLA Pipeline [36] a workflow for the collection, storage, annotation, analysis and exploitation of multimodal data for supporting learning. We used three existing component implementations of the MMLA Pipeline.…”

Section: Methodsmentioning

confidence: 99%

“…This device was not integrated with LearningHub but saved on the local memory of the SimPad, hence transferred via USB to the computer used for the analysis. The SimPad sessions were then manually synchronised with the help of the Visual Inspection Tool (see Section 4.6) [36]. The most sensitive data, the video recording of the participants, were only included during the annotation phase, exclusively by the research team.…”

Section: Methodsmentioning

confidence: 99%

“…The CPR annotations were later synchronised with the sessions using the Visual Inspection Tool (VIT) [36]. VIT allows the manual and semi-automatic annotation of psychomotor learning tasks which can be captured with a set of sensors.…”

Section: Methodsmentioning

confidence: 99%

“…Moreover, to the best of our knowledge, no standardised procedures exist in this field. In this study, we use an emerging approach, the Multimodal Learning Analytics Pipeline (MMLA Pipeline) [36], in the context of CPR training for creating Multimodal Tutor for CPR. We believe that the selected method can be used as road map for the general identification of learners mistakes using multimodal data in authentic training procedures.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Detecting Mistakes in CPR Training with Multimodal Data and Neural Networks

Mitri

Schneider

Specht

et al. 2019

Sensors

Self Cite

View full text Add to dashboard Cite

This study investigated to what extent multimodal data can be used to detect mistakes during Cardiopulmonary Resuscitation (CPR) training. We complemented the Laerdal QCPR ResusciAnne manikin with the Multimodal Tutor for CPR, a multi-sensor system consisting of a Microsoft Kinect for tracking body position and a Myo armband for collecting electromyogram information. We collected multimodal data from 11 medical students, each of them performing two sessions of two-minute chest compressions (CCs). We gathered in total 5254 CCs that were all labelled according to five performance indicators, corresponding to common CPR training mistakes. Three out of five indicators, CC rate, CC depth and CC release, were assessed automatically by the ReusciAnne manikin. The remaining two, related to arms and body position, were annotated manually by the research team. We trained five neural networks for classifying each of the five indicators. The results of the experiment show that multimodal data can provide accurate mistake detection as compared to the ResusciAnne manikin baseline. We also show that the Multimodal Tutor for CPR can detect additional CPR training mistakes such as the correct use of arms and body weight. Thus far, these mistakes were identified only by human instructors. Finally, to investigate user feedback in the future implementations of the Multimodal Tutor for CPR, we conducted a questionnaire to collect valuable feedback aspects of CPR training.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Detecting Mistakes in CPR Training with Multimodal Data and Neural Networks

Mitri

Schneider

Specht

et al. 2019

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

“…The IDM Haptic feedback , Object enrichment and Auditory feedback are implemented to provide feedback on procedural information while the IDM Animation provides supportive information such as speed and path, on the learning task. Summative feedback is provided by collecting, visualizing, and comparing learner’s data with the expert’s data by using the Visual inspection tool [21]. More details on the implementation of these IDMs are provided in the following sections.…”

Section: Introductionmentioning

confidence: 99%

Can You Ink While You Blink? Assessing Mental Effort in a Sensor-Based Calligraphy Trainer

Limbu

Jarodzka

Klemke

et al. 2019

Sensors

Self Cite

View full text Add to dashboard Cite

Sensors can monitor physical attributes and record multimodal data in order to provide feedback. The application calligraphy trainer, exploits these affordances in the context of handwriting learning. It records the expert’s handwriting performance to compute an expert model. The application then uses the expert model to provide guidance and feedback to the learners. However, new learners can be overwhelmed by the feedback as handwriting learning is a tedious task. This paper presents the pilot study done with the calligraphy trainer to evaluate the mental effort induced by various types of feedback provided by the application. Ten participants, five in the control group and five in the treatment group, who were Ph.D. students in the technology-enhanced learning domain, took part in the study. The participants used the application to learn three characters from the Devanagari script. The results show higher mental effort in the treatment group when all types of feedback are provided simultaneously. The mental efforts for individual feedback were similar to the control group. In conclusion, the feedback provided by the calligraphy trainer does not impose high mental effort and, therefore, the design considerations of the calligraphy trainer can be insightful for multimodal feedback designers.

show abstract

A review on data fusion in multimodal learning analytics and educational data mining

Chango

Lara²,

Cerezo

et al. 2022

WIREs Data Min & Knowl

View full text Add to dashboard Cite

The new educational models such as smart learning environments use of digital and context-aware devices to facilitate the learning process. In this new educational scenario, a huge quantity of multimodal students' data from a variety of different sources can be captured, fused, and analyze. It offers to researchers and educators a unique opportunity of being able to discover new knowledge to better understand the learning process and to intervene if necessary. However, it is necessary to apply correctly data fusion approaches and techniques in order to combine various sources of multimodal learning analytics (MLA). These sources or modalities in MLA include audio, video, electrodermal activity data, eye-tracking, user logs, and click-stream data, but also learning artifacts and more natural human signals such as gestures, gaze, speech, or writing. This survey introduces data fusion in learning analytics (LA) and educational data mining (EDM) and how these data fusion techniques have been applied in smart learning. It shows the current state of the art by reviewing the main publications, the main type of fused educational data, and the data fusion approaches and techniques used in EDM/LA, as well as the main open problems, trends, and challenges in this specific research area.

show abstract

Read Between the Lines

Cited by 35 publications

References 32 publications

Detecting Mistakes in CPR Training with Multimodal Data and Neural Networks

Detecting Mistakes in CPR Training with Multimodal Data and Neural Networks

Can You Ink While You Blink? Assessing Mental Effort in a Sensor-Based Calligraphy Trainer

A review on data fusion in multimodal learning analytics and educational data mining

Contact Info

Product

Resources

About