Measuring task engagement as an input to physiological computing

Fairclough, Stephen H.; Ewing, Katie; Roberts, Jenna

doi:10.1109/acii.2009.5349483

Cited by 28 publications

(18 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Extending previous results (eg, Fairclough et al ., ; Marshall, ), and coinciding with more recent findings (eg, Di Mitri et al , ; Junokas et al , ; Spikol et al , ), our findings suggest that although individual modalities can be a good proxy for performance and effort, fusing features from different modalities has the potential to further increase prediction accuracy. In other words, and in line with Giannakos et al (), it is confirmed that data fusion produces more consistent and accurate predictions than those from individual data sources.…”

Section: Discussionsupporting

confidence: 91%

“…Previous studies revealed significant findings in terms of what physiological data are appropriate for explaining students' behavior (Lane & D'Mello, 2019) and modeling and predicting their emotions, engagement with the tasks and performance, in diverse learning settings (D'Mello et al, 2009;Di Mitri et al, 2018;Fairclough et al, 2009;Marshall, 2002;Spikol et al, 2018). In these settings, the exploitation of ML techniques was proposed to reduce human workload during the analysis of learners' interactions, and to select appropriate multimodal data for capturing learners' behaviors, (Andrade et al, 2016;Ochoa et al, 2018).…”

Section: Discussionmentioning

confidence: 99%

“…Gaze has long been studied as an approach for understanding users' behaviors and cognitive states (Lai et al , ). Moreover, Fairclough, Moores, Ewing, and Roberts () found that electroencephalography (EEG) variables were sensitive to disengagement due to cognitive load. Furthermore, effort‐related cardiovascular responses can be mapped to success importance until a maximum effort has been achieved (Wright & Kirby, ).…”

Section: Related Work: Utilizing Multimodal Data To Predict Learning mentioning

confidence: 99%

See 2 more Smart Citations

Building pipelines for educational data using AI and multimodal analytics: A “grey‐box” approach

Sharma¹,

Papamitsiou²,

Giannakos³

2019

Brit J Educational Tech

View full text Add to dashboard Cite

Students' on‐task engagement during adaptive learning activities has a significant effect on their performance, and at the same time, how these activities influence students' behavior is reflected in their effort exertion. Capturing and explaining effortful (or effortless) behavior and aligning it with learning performance within contemporary adaptive learning environments, holds the promise to timely provide proactive and actionable feedback to students. Using sophisticated machine learning (ML) algorithms and rich learner data, facilitates inference‐making about several behavioral aspects (including effortful behavior) and about predicting learning performance, in any learning context. Researchers have been using ML methods in a “black‐box” approach, ie, as a tool where the input data is the learner data and the output is a given class from the chosen construct. This work proposes a methodological shift from the “black‐box” approach to a “grey‐box” approach that bridges the hypothesis/literature‐driven (feature extraction) “white‐box” approach with the computation/data‐driven (feature fusion) “black‐box” approach. This will allow us to utilize data features that are educationally and contextually meaningful. This paper aims to extend current methodological paradigms, and puts into practice the proposed approach in an adaptive self‐assessment case study taking advantage of new, cutting‐edge, interdisciplinary work on building pipelines for educational data, using innovative tools and techniques. What is already known about this topic Capturing and measuring learners' engagement and behavior using physiological data has been explored during the last years and exhibits great potential. Effortless behavioral patterns commonly exhibited by learners, such as “cheating,” “guessing” or “gaming the system” counterfeit the learning outcome. Multimodal data can accurately predict learning engagement, performance and processes. What this paper adds Generalizes a methodology for building machine learning pipelines for multimodal educational data, using a modularized approach, namely the “grey‐box” approach. Showcases that fusion of eye‐tracking, facial expressions and arousal data provide the best prediction of effort and performance in adaptive learning settings. Highlights the importance of fusing data from different channels to obtain the most suited combinations from the different multimodal data streams, to predict and explain effort and performance in terms of pervasiveness, mobility and ubiquity. Implications for practice and/or policy Learning analytics researchers shall be able to use an innovative methodological approach, namely the “grey‐box,” to build machine learning pipelines from multimodal data, taking advantage of artificial intelligence capabilities in any educational context. Learning design professionals shall have the opportunity to fuse specific features of the multimodal data to drive the interpretation of learning outcomes in terms of physiological learner states. The constraints from th...

show abstract

Section: Discussionsupporting

confidence: 91%

Section: Discussionmentioning

confidence: 99%

Section: Related Work: Utilizing Multimodal Data To Predict Learning mentioning

confidence: 99%

See 1 more Smart Citation

Building pipelines for educational data using AI and multimodal analytics: A “grey‐box” approach

Sharma¹,

Papamitsiou²,

Giannakos³

2019

Brit J Educational Tech

View full text Add to dashboard Cite

show abstract

“…Future work should explore further dimensions of driver states and boredom levels. For example, electroencephalography (EEG) has previously been used to generate indices of cognitive engagement (Pope et al 1995; Hassib et al 2017 , which is a facet of task engagement (Fairclough et al 2009 The gamified condition led to slower reactions during a sudden hazard event in the middle of a coasting challenge. This could be attributed to the high cognitive workload demanded to deal with both situations simultaneously, and to the many short off road glances suggesting visual attention was divided between the road and the smartphone application during challenges.…”

Section: Resultsmentioning

confidence: 99%

“…The authors describe it as the quantity and quality of mental resources that promote effortful striving directed toward task goals, i.e., they talk about mental resources that allow us to be attentive or otherwise involved in an undertaking. Fairclough et al (2009) augment this definition by pointing out three distinguishing facets of the construct: cognitive activity, motivational orientation, and affect. The authors reaffirm that the cognitive activity can be described as mental effort, which in turn is conceptualised as energy mobilisation in the service of cognitive goals.…”

Section: Defining Task Engagementmentioning

confidence: 99%