Gaze-based interest detection on newspaper articles

Jacob, Soumy; Ishimaru, Shoya; Bukhari, Syed Saqib; Dengel, Andreas

doi:10.1145/3208031.3208034

Cited by 9 publications

(9 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Demberg (2013) have also recently reported changes in pupillometry due to linguistically induced cognitive load (e.g., comprehending syntactically demanding sentences). Other recent work has also examined user state related changes in pupil diameter in lab-settings such as variations in valence and arousal (Kassem et al, 2017) and interest in real-time (Jacob et al, 2018).…”

Section: Psychophysiological Measures To Assess Cognitive Statesmentioning

confidence: 99%

A Review of Psychophysiological Measures to Assess Cognitive States in Real-World Driving

Lohani

Payne

Strayer

2019

Front. Hum. Neurosci.

241

181

View full text Add to dashboard Cite

As driving functions become increasingly automated, motorists run the risk of becoming cognitively removed from the driving process. Psychophysiological measures may provide added value not captured through behavioral or self-report measures alone. This paper provides a selective review of the psychophysiological measures that can be utilized to assess cognitive states in real-world driving environments. First, the importance of psychophysiological measures within the context of traffic safety is discussed. Next, the most commonly used physiology-based indices of cognitive states are considered as potential candidates relevant for driving research. These include: electroencephalography and event-related potentials, optical imaging, heart rate and heart rate variability, blood pressure, skin conductance, electromyography, thermal imaging, and pupillometry. For each of these measures, an overview is provided, followed by a discussion of the methods for measuring it in a driving context. Drawing from recent empirical driving and psychophysiology research, the relative strengths and limitations of each measure are discussed to highlight each measures' unique value. Challenges and recommendations for valid and reliable quantification from lab to (less predictable) real-world driving settings are considered. Finally, we discuss measures that may be better candidates for a near real-time assessment of motorists' cognitive states that can be utilized in applied settings outside the lab. This review synthesizes the literature on in-vehicle psychophysiological measures to advance the development of effective human-machine driving interfaces and driver support systems.

show abstract

Section: Psychophysiological Measures To Assess Cognitive Statesmentioning

confidence: 99%

A Review of Psychophysiological Measures to Assess Cognitive States in Real-World Driving

Lohani

Payne

Strayer

2019

Front. Hum. Neurosci.

241

181

View full text Add to dashboard Cite

show abstract

“…Two separate feature calculations proposed by Jacob et al [21], [22] are followed, and the concatenation of the future dimensions was performed to investigate the effect of the sensor fusion.…”

Section: A Manual Feature Extraction-based Interest Detectionmentioning

confidence: 99%

“…In our previous two studies, data from the eye tracker and Empatica E4 wristband were separately evaluated using conventional machine learning methods by manually extracting features from gaze data [21], and physiological signals [22]. The fundamental objective of this study is to have different degrees of interest identified using the raw attributes of the eye tracker and the Empatica E4 wristband for the same recorded data.…”

Section: Introductionmentioning

confidence: 99%

Multimodal Assessment of Interest Levels in Reading: Integrating Eye-Tracking and Physiological Sensing

Santhosh,

Dzsotjan,

Ishimaru

2023

IEEE Access

Self Cite

View full text Add to dashboard Cite

Interest in the context of reading holds special significance as it serves as a driving force for learning and education. By understanding and leveraging students' interests, educators can create more effective and enjoyable learning environments that promote personalized learning experiences, enhanced comprehension, deep understanding, and motivation. A multimodal approach integrating gaze and physiological data could provide a more comprehensive and accurate assessment of interest levels. The goal of this study is to measure the level of interest experienced by users when reading newspaper articles by integrating gaze data and physiological responses. An experiment was conducted which recorded the gaze and physiological data from 13 university students reading 18 newspaper articles collected from the BBC news database. An SMI eye-tracker and an Empatica E4 wristband were used synchronously to capture the user's eye movements and physiological data. To predict the interest levels of the participants, a manual feature extraction-based approach and a deep learning-based approach were employed. The interest levels were divided into four-class and binary based on the responses from the participants. A CNN-LSTM model using the gaze features outperformed other models in terms of accuracy and F1-score with 52.8% and 51.8 for four-class and 82.3% and 81.7 for binary classification using leave-one-document-out and leave-oneparticipant-out cross-validation, respectively.INDEX TERMS Affective-computing, Eye-tracking, Physiological sensing, Reading analysis.

show abstract

“…Based on data from a user study, they suggested a two-staged reading model for explaining the cognitive processes inherent in relevance judgements. Jacob et al (2018) investigated whether eye movements can be used to infer the interest of a reader in a currently read article. Bhattacharya et al (2020b) encoded fixations from participants' scanpaths over documents from the g-REL corpus and trained a convolutional neural network (CNN) with the perceived relevance as prediction target.…”

Section: Relevance Estimation From Reading Behaviormentioning

confidence: 99%

Implicit Estimation of Paragraph Relevance From Eye Movements

Barz

Bhatti

Sonntag

2022

Front. Comput. Sci.

View full text Add to dashboard Cite

Eye movements were shown to be an effective source of implicit relevance feedback in constrained search and decision-making tasks. Recent research suggests that gaze-based features, extracted from scanpaths over short news articles (g-REL), can reveal the perceived relevance of read text with respect to a previously shown trigger question. In this work, we aim to confirm this finding and we investigate whether it generalizes to multi-paragraph documents from Wikipedia (Google Natural Questions) that require readers to scroll down to read the whole text. We conduct a user study (n = 24) in which participants read single- and multi-paragraph articles and rate their relevance at the paragraph level with respect to a trigger question. We model the perceived document relevance using machine learning and features from the literature as input. Our results confirm that eye movements can be used to effectively model the relevance of short news articles, in particular if we exclude difficult cases: documents which are on topic of the trigger questions but irrelevant. However, our results do not clearly show that the modeling approach generalizes to multi-paragraph document settings. We publish our dataset and our code for feature extraction under an open source license to enable future research in the field of gaze-based implicit relevance feedback.

show abstract

Gaze-based interest detection on newspaper articles

Cited by 9 publications

References 23 publications

A Review of Psychophysiological Measures to Assess Cognitive States in Real-World Driving

A Review of Psychophysiological Measures to Assess Cognitive States in Real-World Driving

Multimodal Assessment of Interest Levels in Reading: Integrating Eye-Tracking and Physiological Sensing

Implicit Estimation of Paragraph Relevance From Eye Movements

Contact Info

Product

Resources

About