Expertise estimation based on simple multimodal features

Ochôa, Xavier; Chiluiza, Katherine; Méndez, Gonzalo Gabriel; Luzardo, Gonzalo; Guamán, Bruno; Castells, James

doi:10.1145/2522848.2533789

Cited by 55 publications

(57 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…This finding regarding slower average writing speed conflicts with Ochoa et al's previous report [7]. However, Ochoa and colleagues' work did not conduct any actual empirical analyses of expert behavior, and the automated analyses conducted only evaluated low-level geometric primitives of strokes (i.e., lines, circles).…”

Section: Discussionmentioning

confidence: 71%

“…In other directly related work based on the Math Data Corpus, Ochoa et al [7] reported that pen stroke writing speed is a predictor of math expertise:…”

Section: Related Research Findingsmentioning

confidence: 99%

“…Analysis of students' writing behavior potentially can play a valuable role in identifying differences among students in domain expertise [7,11], especially in areas like mathematics that include a great deal of spatial content like symbols and diagrams. The present research aims to develop automatic prediction of domain expertise in math by analyzing pen stroke signal features, such as number of strokes, stroke distance, duration, speed, and pressure.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Combining empirical and machine learning techniques to predict math expertise using pen signal features

Zhou

Hang

Oviatt³

et al. 2014

Proceedings of the 2014 ACM Workshop on Multimodal Learning Analytics Workshop and Grand Challenge

View full text Add to dashboard Cite

Multimodal learning analytics aims to automatically analyze students' natural communication patterns based on speech, writing, and other modalities during learning activities. This research used the Math Data Corpus, which contains timesynchronized multimodal data from collaborating students as they jointly solved problems varying in difficulty. The aim was to investigate how reliably pen signal features, which were extracted as students wrote with digital pens and paper, could identify which student in a group was the dominant domain expert. An additional aim was to improve prediction of expertise based on joint bootstrapping of empirical science and machine learning techniques. To accomplish this, empirical analyses first identified which data partitioning and pen signal features were most reliably associated with expertise. Then alternative machine learning techniques compared classification accuracies based on all pen features, versus empirically selected ones. The best unguided classification accuracy was 70.8%, which improved to 83.3% with empirical guidance. These results demonstrate that handwriting signal features can predict domain expertise in math with high reliability. Hybrid methods also can outperform blackbox machine learning in both accuracy and transparency.

show abstract

Section: Discussionmentioning

confidence: 71%

“…In other directly related work based on the Math Data Corpus, Ochoa et al [7] reported that pen stroke writing speed is a predictor of math expertise:…”

Section: Related Research Findingsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Combining empirical and machine learning techniques to predict math expertise using pen signal features

Zhou

Hang

Oviatt³

et al. 2014

Proceedings of the 2014 ACM Workshop on Multimodal Learning Analytics Workshop and Grand Challenge

View full text Add to dashboard Cite

show abstract

“…Ochoa et al (2013) report on successfully using handwriting speed (using a digital pen) to distinguish between experts and nonexperts in a collaborative environment solving mathematical problems. This work underlines the usefulness of such low-level features in solving classification problems.…”

Section: Prior Workmentioning

confidence: 99%

Using Keystroke Analytics to Improve Pass-Fail Classifiers

Casey

2017

Learning Analytics

View full text Add to dashboard Cite

ABSTRACT. Learning analytics offers insights into student behaviour and the potential to detect poor performers before they fail exams. If the activity is primarily online (for example computer programming), a wealth of low-level data can be made available that allows unprecedented accuracy in predicting which students will pass or fail. In this paper, we present a classification system for early detection of poor performers based on student effort data, such as the complexity of the programs they write, and show how it can be improved by the use of low-level keystroke analytics.

show abstract

“…Working on co-located, collaborative problem solving, Ochoa et al [28] used video, audio and pen stroke information, extracting simple features to discriminate between experts and non-experts. More recently, other authors have proposed Feeler [12], a system that uses EEG in conjunction with application logs to promote student reflection about learning.…”

Section: Multimodal Learning and Teaching Analyticsmentioning

confidence: 99%

Teaching analytics

Prieto

Sharma

Dillenbourg

et al. 2016

Proceedings of the Sixth International Conference on Learning Analytics &Amp; Knowledge - LAK '16

View full text Add to dashboard Cite

ABSTRACT'Teaching analytics' is the application of learning analytics techniques to understand teaching and learning processes, and eventually enable supportive interventions. However, in the case of (often, half-improvised) teaching in face-to-face classrooms, such interventions would require first an understanding of what the teacher actually did, as the starting point for teacher reflection and inquiry. Currently, such teacher enactment characterization requires costly manual coding by researchers. This paper presents a case study exploring the potential of machine learning techniques to automatically extract teaching actions during classroom enactment, from five data sources collected using wearable sensors (eye-tracking, EEG, accelerometer, audio and video). Our results highlight the feasibility of this approach, with high levels of accuracy in determining the social plane of interaction (90%, κ=0.8). The reliable detection of concrete teaching activity (e.g., explanation vs. questioning) accurately still remains challenging (67%, κ=0.56), a fact that will prompt further research on multimodal features and models for teaching activity extraction, as well as the collection of a larger multimodal dataset to improve the accuracy and generalizability of these methods.

show abstract

Expertise estimation based on simple multimodal features

Cited by 55 publications

References 20 publications

Combining empirical and machine learning techniques to predict math expertise using pen signal features

Combining empirical and machine learning techniques to predict math expertise using pen signal features

Using Keystroke Analytics to Improve Pass-Fail Classifiers

Teaching analytics

Contact Info

Product

Resources

About