The impact of on‐demand metacognitive help on effortful behaviour: A longitudinal study using task‐related visual analytics

Abstract

This longitudinal study investigates the differences in learners' effortful behaviour over time due to receiving metacognitive help-in the form of on-demand taskrelated visual analytics. Specifically, learners' interactions (N = 67) with the tasks were tracked during four self-assessment activities, conducted at four discrete points in time, over a period of 8 weeks. The considered and coded time points were: (a) prior to providing the metacognitive help; (b) while the task-related visual analytics were available (treatment); (c) after the removal of the treatment; and (d) while the option to receive metacognitive help was available again. To measure learners' effortful behaviour across the self-assessment activities, this study utilized learners' responsetimes to correctly/wrongly complete the tasks and on-task effort expenditure. The panel data analysis shown that the usage of metacognitive help caused statistically significant changes in learners' effortful behaviour, mostly in the third and fourth phase. Statistically significant changes were detected also in the usage of metacognitive help. These results provide empirical evidence on the benefits of taskrelated visual analytics to support learners' on-task engagement, and suggest relevant cues on how metacognitive help could be designed and prompted by focusing on the "task", instead of the "self".

Using learning analytics to measure self‐regulated learning: A systematic review of empirical studies in higher education

“…The study in (Ye & Pennisi, 2022) used the number of late submission as an indicator of procrastination, while the study in (Ilves et al, 2018) captures anti‐procrastination from two angles using two indicators: the number of days that a student started working on the tasks after they were released, and the average number of days of submission before the deadlines. The studies in (Li et al, 2020) (Papamitsiou & Economides, 2021) (Rodriguez et al, 2021) (Feldman‐Maggor et al, 2022) measured anti‐procrastination based on the proportion of materials studied or tasks completed before the deadlines. Instead of studying procrastination per task (micro level), the study in (Saqr et al, 2019) investigates that on the course level (macro level).…”

“…Different approaches were used to capture this skill based on indicators that represent students' behaviours to monitor and get feedback about their performance. The studies in (Pardo et al, 2017) and (Papamitsiou & Economides, 2021) used the number of times students accessed to dashboard analytics visualizations as an indicator of students' self‐monitoring behaviour. Dashboards are tools that visualize learners' trace data and performance in order to improve their self‐awareness of their learning.…”

“…Dashboards are tools that visualize learners' trace data and performance in order to improve their self‐awareness of their learning. The study in (Pardo et al, 2017) used only the total number of times accessing the dashboard, whereas the study in (Papamitsiou & Economides, 2021) used the total number of times and the average time students spent viewing the dashboard. Another approach was used by the study in (Iraj et al, 2021) where the study aimed to capture students access to the feedback sent to them by embedding links in the feedback messages.…”

“…Instead, they assume that academic performance is a consequence of self‐regulation, and focused only on evaluation the relation between these indicators and students' performance. Likewise, the study in (Papamitsiou & Economides, 2021) used students' performance in self‐assessment, instead of academic performance, to evaluate the predictability of the indicators used. The study in (Pardo et al, 2017) found that combining trace data indicators with data collected using Motivational Strategies for Learning Questionnaire (MSLQ) improved the prediction of students' academic performance.…”

Cognitive psychology and the classroom: what teachers need to know

“…Freud posited that the subconscious inundates the learner's conscious mind with conflicting forces from the id (the primitive and instinctual part of the mind) and the superego (the moral conscience). The ego, situated at the conscious level, is a facet teachers can use to extract features of learner performance like conscious awareness, cognitive engagement, metacognitive involvement, and information-processing ability (Papamitsiou et al, 2021). A fundamental tenet of Freudian theory is the significant role attributed to the unconscious mind in shaping behaviour (Cherry, 2018).…”