Effects of text difficulty and readers on predicting reading comprehension from eye movements

Copeland, Leana; Gedeon, Tom; Caldwell, Sabrina

doi:10.1109/coginfocom.2015.7390628

Cited by 12 publications

(7 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Regression fixations were any fixations that were on a word with an index lower than that of the previous word that was fixated on. These features have been used in similar modeling efforts (Copeland, 2016;Copeland & Gedeon, 2013;Copeland et al, 2014Copeland et al, , 2015Mart ınez-G omez & Aizawa, 2014) and have been empirically linked to factors that influence text comprehension, including mind-wandering (e.g., Reichle et al, 2010;Uzzaman & Joordens, 2011) and text difficulty (Rayner et al, 2006). Although it is not an eye movement feature, we also included page reading time in seconds as it is perhaps the most ubiquitous dependent measure in reading research and has been linked to comprehension (e.g., Mills, Graesser, Risko, & D'Mello, 2017).…”

Section: Eye Movement Featuresmentioning

confidence: 99%

“…For example, Copeland, Gedeon, and Mendis (2014) had participants read text presented on slides (as in a slide deck). They trained artificial neural networks on several eye movement features computed for each slide (e.g., number of fixations, mean fixation duration, total text fixation duration, number of regressions) to generate predictions of comprehension as assessed by performance on questions presented alongside or immediately after each slide (see also Copeland, 2016; Copeland & Gedeon, 2013; Copeland, Gedeon, & Caldwell, 2015). Similarly, Martínez‐Gómez and Aizawa (2014) achieved above‐chance predictions of binarized (high vs. low) comprehension for short (~450 word) educational texts using a combination of linguistic features (e.g., word length) and eye movement features, with the latter being more discriminative (see also Lou, Liu, Kaakinen, & Li, 2017 for models predicting language skill).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

What Eye Movements Reveal About Later Comprehension of Long Connected Texts

et al. 2020

View full text Add to dashboard Cite

We know that reading involves coordination between textual characteristics and visual attention, but research linking eye movements during reading and comprehension assessed after reading is surprisingly limited, especially for reading long connected texts. We tested two competing possibilities: (a) the weak association hypothesis: Links between eye movements and comprehension are weak and short-lived, versus (b) the strong association hypothesis: The two are robustly linked, even after a delay. Using a predictive modeling approach, we trained regression models to predict comprehension scores from global eye movement features, using participant-level crossvalidation to ensure that the models generalize across participants. We used data from three studies in which readers (Ns = 104, 130, 147) answered multiple-choice comprehension questions 30 min after reading a 6,500-word text, or after reading up to eight 1,000-word texts. The models generated accurate predictions of participants' text comprehension scores (correlations between observed and predicted comprehension: 0.384, 0.362, 0.372, ps < .001), in line with the strong association hypothesis. We found that making more, but shorter fixations, consistently predicted comprehension across all studies. Furthermore, models trained on one study's data could successfully predict comprehension on the others, suggesting generalizability across studies. Collectively, these findings suggest that there is a robust link between eye movements and subsequent comprehension of a long connected text, thereby connecting theories of low-level eye movements with those of higher order text processing during reading.

show abstract

Section: Eye Movement Featuresmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

What Eye Movements Reveal About Later Comprehension of Long Connected Texts

et al. 2020

View full text Add to dashboard Cite

show abstract

“…They used artificial neural networks to predict performance on the comprehension questions from multiple global eye-movement measures (e.g., average fixation duration), with 79-89% accurate classification rate (correct, half-correct, incorrect response). Although this type of method does not allow for a clear links to be established between specific eye-movement measures and comprehension, the results do suggest that eye movements can be used to successfully predict comprehension scores (see also Copeland, Gedeon, & Caldwell, 2016;Copeland & Gedeon, 2013;Martínez-Gómez & Aizawa, 2014). Inhoff, Gregg, and Radach (2018) investigated the predictive relationship between subsets of eye movements and comprehension ability more directly.…”

Section: Predicting Comprehension Accuracy From Eye-movement Behaviourmentioning

confidence: 99%

Using Eye-Tracking Measures to Predict Reading Comprehension

Mézière

Reichle³

et al. 2021

Preprint

View full text Add to dashboard Cite

Research on reading comprehension assessments suggests that they measure overlapping but not identical cognitive skills. In this paper, we examined the potential of eye-tracking as a tool for assessing reading comprehension. We administered three widely-used reading comprehension tests with varying task demands to 79 typical adult readers while monitoring their eye movements. In the York Assessment for Reading Comprehension (YARC), participants were given passages of text to read silently, followed by comprehension questions. In the Gray Oral Reading Test (GORT-5), participants were given passages of text to read aloud, followed by comprehension questions. In the sentence comprehension subtest of the Wide Range Achievement Test (WRAT-4), participants were given sentences with a missing word to read silently, and had to provide the missing word (i.e., a cloze task). Results from linear models predicting comprehension scores from eye-tracking measures yielded different patterns of results between the three tests. Models with eye-tracking measures always explained significantly more variance compared to baseline models with only reading speed, with R-squared 4 times higher for the YARC, 3 times for the GORT, and 1.3 times for the WRAT. Importantly, despite some similarities between the tests, no common good predictor of comprehension could be identified across the tests. Overall, the results suggest that reading comprehension tests do not measure the same cognitive skills to the same extent, and that participants adapted their reading strategies to the tests’ varying task demands. Finally, this study suggests that eye-tracking may provide a useful alternative for measuring reading comprehension.

show abstract

“…The emergence of e-learning platforms has led to the development of more unobtrusive, objective and reliable methods for gathering data, using the logging capabilities afforded by these platforms (Cocea and Weibelzahl, 2011). Different data sources can be instrumented and monitored, including learners' clickstreams (Siemens, 2013), eyes movements (Copeland et al, 2015), participation (Xing et al, 2015), and/or assessments (Fidalgo-Blanco et al, 2015;Snodgrass-Rangel et al, 2015). During the data analysis process, the captured data undergo different transformations, in order to be finally translated into understandable and exploitable human knowledge.…”

Section: Monitoring E-learningmentioning

confidence: 99%

Leveraging learners' activity logs for course reading analytics using session-based indicators

Sadallah

Encelle

Maredj

et al. 2020

IJTEL

View full text Add to dashboard Cite

A challenge that course authors face when reviewing their contents is to detect how to improve their courses in order to meet the expectations of their learners. In this paper, we propose an analytical approach that exploits learners' logs of reading to provide authors with insightful data about the consumption of their courses. We first model reading activity using the concept of reading-session and propose a new and efficient session identification. We then elaborate a list of indicators computed using learners' reading sessions that allow to represent their behaviour and to infer their needs. We evaluate our proposals with course authors and learners using logs from a major e-learning platform. Interesting results were found. This demonstrates the effectiveness of the approach in identifying aspects and parts of a course that may prevent it from being easily read and understood, and for guiding the authors through the analysis and review tasks.

show abstract

Effects of text difficulty and readers on predicting reading comprehension from eye movements

Cited by 12 publications

References 24 publications

What Eye Movements Reveal About Later Comprehension of Long Connected Texts

What Eye Movements Reveal About Later Comprehension of Long Connected Texts

Using Eye-Tracking Measures to Predict Reading Comprehension

Leveraging learners' activity logs for course reading analytics using session-based indicators

Contact Info

Product

Resources

About