Are There Distinctive Profiles in Examinee Essay‐Writing Processes?

Abstract

Grouping individuals according to a set of measured characteristics, or profiling, is frequently used in describing, understanding, and acting on a phenomenon. The advent of computer‐based assessment offers new possibilities for profiling writing because aspects can be captured that were not heretofore observable. We explored whether writing processes could be profiled of over 30,000 adults taking a high‐school equivalency examination. Process features were extracted from keystroke logs, aggregated into composite indicators, and used with essay score to assign individuals to profiles. Analyses included computing the percentages of individuals that could be classified, using MANOVA to examine differences among profiles on external variables, and examining if profiles could be distinguished from one another based on patterns derived from cluster analysis. Results showed that about 30% of examinees could be classified into profiles that were largely distinct. These results contribute toward a foundation for using such profiles in describing how individuals compose and in how their writing might be improved.

The use of process data in large-scale assessments: a literature review

“…In this category, the studies involved writing assessments and examinees' interactions with them. These studies were published between 2008 (Crawford et al, 2008) and 2022 (Bennett et al, 2022;Gong et al, 2022;Talebinamvar & Zarrabi, 2022). Most studies relied on some theoretical model of writing.…”

Assessing Japanese junior high school students’ English achievement through computer-based testing in the classroom: a case of integrated reading-into-writing continuous task

“…Few studies have developed questions that take advantage of the CBT features and analyzed the test-takers' writing. In recent years, many studies have attempted to elucidate learners' writing process by analyzing keystrokes obtained by having them type English on a computer (e.g., Bennett et al, 2022;Talebinamvar & Zarrabi, 2022). These studies, however, are rarely intended to provide teachers with a simple way to identify students' obstacles in the process leading up to the product or connect them to classroom instruction.…”