Understanding the keystroke log: the effect of writing task on keystroke features

Abstract

Keystroke logging is used to automatically record writers' unfolding typing process and to get insight into moments when they struggle composing text. However, it is not clear which and how features from the keystroke log map to higher-level cognitive processes, such as planning and revision. This study aims to investigate the sensitivity of frequently used keystroke features across tasks with different cognitive demands. Two keystroke datasets were analyzed: one consisting of a copy task and an email writing task, and one with a larger difference in cognitive demand: a copy task and an academic summary task. The differences across tasks were modeled using Bayesian linear mixed effects models. Posterior distributions were used to compare the strength and direction of the task effects across features and datasets. The results showed that the average of all interkeystroke intervals were found to be stable across tasks. Features related to the time between words and (sub)sentences only differed between the copy and the academic task. Lastly, keystroke features related to the number of words, revisions, and total time, differed across tasks in both datasets. To conclude, our results indicate that the latter features are related to cognitive load or task complexity. In addition, our research shows that keystroke features are sensitive to small differences in the writing tasks at hand.

Identifying Mixture Components From Large-Scale Keystroke Log Data

“…In addition, the fluency effect in this research is consistent across writing genres. It echoes back to Conijn et al’s (2019) finding, that is, the features, particularly for within words, are robust across writing tasks.…”

“…As shown in Table 7 , the correlations related to the lower-level cognitive process component could tentatively be interpreted as the fluency effect , and modeling the WithinWord linguistic context data might reflect a measure of dysfluency with orthography. Orthography is the least demanding of the measured cognitive processes ( Deane et al, 2008 ; Deane et al, 2011 ; Conijn et al, 2019 ). If an 8 th grade student has too many pauses at the location of typing a word ( WithinWord pause data), it is more likely that he/she has difficulty spelling rather than that he/she is instead engaging with a higher-level cognitive process (e.g., word choice or intention reflection).…”

“…Most of the relevant studies have employed descriptive statistics or data reduction techniques. For example, Conijn et al (2019) found that properties including a written essay’s length, the total time spent on completing the essay, and the frequency of revision, differed across distinct writing tasks. Medimorec and Risko (2017) examined pause data and found meaningful differences among the pause frequencies at different text boundaries (e.g., between sentences); specifically, the registered number of pauses at word boundaries (e.g., between words) was higher for argumentative essays than for narrative essays.…”

An Overview of Writing Process Research: Using Innovative Tasks and Techniques for a Better Understanding of L2 Writing Processes in Assessment Contexts

“…With the majority of high-stakes language tests becoming computerized, and given the limitations imposed by verbal protocols and questionnaires, an increasing number of L2 process studies started to rely on new techniques such as keystroke logging programs to examine writing process (Almond et al, 2012;Barkaoui, 2016Barkaoui, , 2019Chukharev-Hudilainen, 2019;Conijn et al, 2019;Leijten & Van Waes, 2013;Miller et al, 2008;Révész et al, 2017;Zhang et al, 2017). Unlike verbal protocols, keystroke logging enabled researchers to collect writing process data in a non-invasive and natural context.…”

Early prediction of writing quality using keystroke logging

“…The different tasks might lead to different writing processes, and consequently also different keystroke data. Previous work already showed that keystroke data is sensitive to small differences in writing tasks, such as copy writing versus email writing (Conijn et al, 2019). In addition, differences in source usage, such as independent and integrated tasks, have been shown to have an effect on students' pausing and revision behavior (Michel et al, 2020).…”