Work

Unknown Word Detection for English as a Second Language (ESL) Learners using Gaze and Pre-trained Language Models

Year: 2025

Type: conference-paper

Abstract: English as a Second Language (ESL) learners often encounter unknown words that hinder their text comprehension. Automatically detecting these words as users read can enable computing systems to provide just-in-time definitions, synonyms, or contextual explanations, thereby helping users learn vocabulary in a natural and seamless manner. This paper presents EyeLingo, a transformer-based machine learning method that predicts the probability of unknown words based on text content and eye gaze trajectory in real time with high accuracy. A 20-participant user study revealed that our method can achieve an accuracy of 97.6%, and an F1-score of 71.1%. We implemented a real-time reading assistance prototype to show the effectiveness of EyeLingo. The user study shows improvement in willingness to use and usefulness compared to baseline methods.

Source: CHI Conference on Human Factors in Computing Systems

DOI: 10.1145/3706598.3714181

Authors: Jiexin Ding, Bowen Zhao, Yuntao Wang, Xinyun Liu, Rui Hao, Ishan Chatterjee, Yuanchun Shi

Institutions: University of Washington, Tsinghua University, Rice University, University of Chinese Academy of Sciences, Qinghai University

Language: English

Topic: Second Language Acquisition and Learning

Subfield: Developmental and Educational Psychology

Field: Psychology

Domain: Social Sciences

SDG: Quality education

Open Access status: hybrid

OpenAlex ID: W4409747069

FWCI: 5.516

Cites: 35

Cited by: 1

Related to: 10