GazeReader: Detecting Unknown Word Using Webcam for English as a Second Language (ESL) Learners

Ding, Jiexin; Zhao, Bowen; Huang, Yuqi; Wang, Yuntao; Shi, Yuanchun

doi:10.1145/3544549.3585790

Cited by 10 publications

(4 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Wagner 30 used the eye-tracking headset to detect the impact of the teacher's gaze proxy on the learning effect of the students, but he didn't analyze the relationship between eye movement data and attention, and need to rely on additional equipment. Ding 31 propose GazeReader, an unknown word detection method only using a webcam, and provide a good idea for positioning the sight, but he need to train a more complex language model to capture context information.…”

Section: Research On the Role Of Eye Contact In Speechmentioning

confidence: 99%

a approach of query audience’s attention in virtual speech

kang,

Yang,

Song

et al. 2024

Preprint

View full text Add to dashboard Cite

Eye contact between speakers and audiences can help speakers get better effectiveness. virtual speeches are a very popular way for multi-user to communicate remotely, but it has the disadvantage of being unable to make eye contact. This paper proposes the online audience’s attention evaluation based on gaze tracking. Instead of using less-obtainable and expensive eye-tracking equipment, this article only uses webcams to capture audience's head pose, fixation time and other features. We also proposed a set of indexes, which can be used to evaluate the audience's degree of attention, making up for the defect that the speaker cannot obtain the audience's concentration through eye contact during online speeches. We selected 96 students and divided them into three groups for a 20-minute curriculum simulation, and used the Spearman correlation coefficient to analyse the correlation between our evaluation indicators and concentration, the result showed that each evaluation index has a significant correlation with the degree of attention (p=0.01), and all the students in the focused group met the thresholds set by each of our evaluation indicators, while the students in the non-focused group failed to reach the standard, the accuracy rate is 100%. In addition, we used EEG signals to verify the accuracy of the system evaluation.During the simulation, eye movement data and EEG signals were measured synchronously for the second group of students. The EEG results of the second group of students were consistent with the results of systematic evaluation.All the experimental work on the ordinary equipment such as a notebook computer which integrated a cpu of 2.30GHZ and a 130W pixel camera without any additional equipment. Our research provides a low-cost method for attention monitoring with reference value across multiple domains. In the field of webpage design, it can enhance page structure and improve user experience based on the distribution of user attention. In the realm of driving safety, real-time monitoring of driver attention can prevent fatigue driving. In the medical field, monitoring the attention of doctors and patients contributes to improving surgical safety. In education, attention monitoring of students allows for adjustments in teaching methods, ultimately enhancing the quality of education. In the industrial sector, monitoring worker attention can reduce operational errors and improve overall work efficiency.

show abstract

Section: Research On the Role Of Eye Contact In Speechmentioning

confidence: 99%

a approach of query audience’s attention in virtual speech

kang,

Yang,

Song

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…Reasoning. Gaze has been applied across various tasks related to cognitive reasoning, including collaborative robotics [8,21,38], interactive technology inputs [51,53,69], reading assistant [4,23], cognitive load measurement [50], and intent classification [24,31,45,72]. Eye movements is a reliable indicator of user intent -"eyes rarely visit objects that are irrelevant to the action" [35,62], which suggests a strong correlation between eye movements and inquiry intent in daily scenarios.…”

Section: Gaze-based Intentionmentioning

confidence: 99%

G-VOILA: Gaze-Facilitated Information Querying in Daily Scenarios

Wang,

Shi,

Wang

et al. 2024

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.

Self Cite

View full text Add to dashboard Cite

Modern information querying systems are progressively incorporating multimodal inputs like vision and audio. However, the integration of gaze --- a modality deeply linked to user intent and increasingly accessible via gaze-tracking wearables --- remains underexplored. This paper introduces a novel gaze-facilitated information querying paradigm, named G-VOILA, which synergizes users' gaze, visual field, and voice-based natural language queries to facilitate a more intuitive querying process. In a user-enactment study involving 21 participants in 3 daily scenarios (p = 21, scene = 3), we revealed the ambiguity in users' query language and a gaze-voice coordination pattern in users' natural query behaviors with G-VOILA. Based on the quantitative and qualitative findings, we developed a design framework for the G-VOILA paradigm, which effectively integrates the gaze data with the in-situ querying context. Then we implemented a G-VOILA proof-of-concept using cutting-edge deep learning techniques. A follow-up user study (p = 16, scene = 2) demonstrates its effectiveness by achieving both higher objective score and subjective score, compared to a baseline without gaze data. We further conducted interviews and provided insights for future gaze-facilitated information querying systems.

show abstract

“…Difficult-to-read word detection methods in English have been proposed (2,3) . In these methods, because of low eye-tracking accuracy (eye-tracking error), texts are displayed using a dedicated user interface (2) or additional features are used to improve the detection accuracy (2,3) . In the method of reference (2), the spaces between the rows are clearly wider for identifying the reading row.…”

Section: Introductionmentioning

confidence: 99%

“…However, the range of application of such wide row spacing is limited. Reference (2) and reference (3) used additional features such as the rarity of different words (2,3) . However, the rarity of any given word does not always indicate how difficult it would be to read for all users owing to individual differences in knowledge.…”

Section: Introductionmentioning

confidence: 99%

Detecting Difficult Japanese Characters using Eye-tracking from Multi-row Long Text

Ito,

Ishizawa,

Kageyama

2023

The Proceedings of the 10th International Conference on Intelligent Systems and Image Processing 2023

View full text Add to dashboard Cite

This study aimed to develop a method for detecting difficult-to-read characters by tracking the gazes of personal computer (PC) users. A method for detecting English words that are difficult to read was proposed previously. However, this method required a dedicated user interface because the accuracy of the eye tracker is low (eye-tracking errors). In actual PC work, the range of use of a dedicated user interface is limited, and efficient work may be hindered. In this study, we propose a character detection method with eye-tracking error correction using projective transformation. The usefulness of the proposed method was examined based on the detection accuracy. The results showed that the proposed method was effective in detecting difficult-to-read characters in F-measure > 0.58 in the first row of the text. Moreover, the usefulness of the proposed method was demonstrated for nine rows or shorter texts compared with no-correction eyetracking data.

show abstract

GazeReader: Detecting Unknown Word Using Webcam for English as a Second Language (ESL) Learners

Cited by 10 publications

References 23 publications

a approach of query audience’s attention in virtual speech

a approach of query audience’s attention in virtual speech

G-VOILA: Gaze-Facilitated Information Querying in Daily Scenarios

Detecting Difficult Japanese Characters using Eye-tracking from Multi-row Long Text

Contact Info

Product

Resources

About