Estimation of reading subjective understanding based on eye gaze analysis

Abstract

The integration of ubiquitous technologies in the field of education has considerably enhanced our way of learning. Such technologies enable students to get a gradual feedback about their performance and to provide adapted learning materials. It is particularly important in the domain of foreign language learning which requires intense daily practice. One of the main inputs of adaptive learning systems is the user’s understanding of a reading material. The reader’s understanding can be divided into two parts: the objective understanding and the subjective understanding. The objective understanding can be measured by comprehension questions about the content of the text. The subjective understanding is the reader’s perception of his own understanding. The subjective understanding plays an important role in the reader’s motivation, self-esteem and confidence. However, its automatic estimation remains a challenging task. This paper is one of the first to propose a method to estimate the subjective understanding. We show that using the eye gaze to predict the subjective understanding improves the estimation by 13% as compared to using comprehension questions.

Linking Use Cases and Associated Requirements

“…The experiments conducted only investigated the readers' observable reading behavior without examining their concrete understanding of the content. For this purpose, we follow the work of Sanches et al (2017Sanches et al ( , 2018) who in vestigated the readers' understanding of documents by using subjective comprehension questions and objective eye gaze data. Based on the work of Sanches et al (2017Sanches et al ( , 2018, we want to adapt the experimental design of our eye tracking study to create further replications of our experiment accord ing to the systematic approach for replication of experiments in software engineering by Gómez et al (2014).…”

“…For this purpose, we follow the work of Sanches et al (2017Sanches et al ( , 2018) who in vestigated the readers' understanding of documents by using subjective comprehension questions and objective eye gaze data. Based on the work of Sanches et al (2017Sanches et al ( , 2018, we want to adapt the experimental design of our eye tracking study to create further replications of our experiment accord ing to the systematic approach for replication of experiments in software engineering by Gómez et al (2014). In this way, on the one hand, we increase the sample size to strengthen the conclusion validity of our findings.…”

“…In this way, on the one hand, we increase the sample size to strengthen the conclusion validity of our findings. On the other hand, we collect the necessary information to reuse the analysis ap proach by Sanches et al (2017Sanches et al ( , 2018 that combines subjec tive ratings in form of Likert scales with objective data, used for an Support Vector Regression (SVR) model.…”

What Is the Gaze Behavior of Pedestrians in Interactions with an Automated Vehicle When They Do Not Understand Its Intentions?

“…driver's operations. For this issue, the gaze duration of the pedestrians at an AV has been focused in our study because the gaze duration is related to the levels of understanding [16,17,18,19,20]. We consider that the gaze duration could express pedestrians' requirements of understanding the driving intentions of an AV while also represent a degree of the pedestrian's understanding of the AV's driving intentions.…”

Subjective Difficulty Estimation of Educational Comics Using Gaze Features

“…However, it is not easy to detect which part of the content they feel is difficult, even in e-book style that can collect clickstream data, because some pages include several topics on the same page. Text-based materials have similar problems, and previous research found that gaze features are effective for difficulty estimation [6], [7]. However, because of the characteristics of comics, information consists of illustrations and texts and is contained on one page, so it is complex compared to reading materials consisting of only text.…”