No abstract

Multimodal learning analytics aims to automatically analyze students' natural communication patterns based on speech, writing, and other modalities during learning activities. This research used the Math Data Corpus, which contains timesynchronized multimodal data from collaborating students as they jointly solved problems varying in difficulty. The aim was to investigate how reliably pen signal features, which were extracted as students wrote with digital pens and paper, could identify which student in a group was the dominant domain expert. An additional aim was to improve prediction of expertise based on joint bootstrapping of empirical science and machine learning techniques. To accomplish this, empirical analyses first identified which data partitioning and pen signal features were most reliably associated with expertise. Then alternative machine learning techniques compared classification accuracies based on all pen features, versus empirically selected ones. The best unguided classification accuracy was 70.8%, which improved to 83.3% with empirical guidance. These results demonstrate that handwriting signal features can predict domain expertise in math with high reliability. Hybrid methods also can outperform blackbox machine learning in both accuracy and transparency.

As commercial pen-centric systems proliferate, they create a parallel need for analytic techniques based on dynamic writing. Within educational applications, recent empirical research has shown that signal-level features of students’ writing, such as stroke distance, pressure and duration, are adapted to conserve total energy expenditure as they consolidate expertise in a domain. The present research examined how accurately three different machine-learning algorithms could automatically classify users’ domain expertise based on signal features of their writing, without any content analysis. Compared with an unguided machine-learning classification accuracy of 71%, hybrid methods using empirical-statistical guidance correctly classified 79–92% of students by their domain expertise level. In addition to improved accuracy, the hybrid approach contributed a causal understanding of prediction success and generalization to new data. These novel findings open up opportunities to design new automated learning analytic systems and student-adaptive educational technologies for the rapidly expanding sector of commercial pen systems.