Work

Data Asgardians at BEA 2026 Shared Task 1: A Hybrid Transformer–Feature Ensemble for L1-Aware English Vocabulary Difficulty Prediction

Year: 2026

Type: conference-paper

Abstract: This paper presents our system for the BEA 2026 Shared Task on Vocabulary Difficulty Prediction for English Learners (Felice and Skidmore, 2026).The task requires predicting psychometrically calibrated GLMM difficulty scores for English vocabulary items across three learner first-language (L1) backgrounds: Spanish (ES), German (DE), and Mandarin Chinese (CN).Our approach studies how hand-crafted linguistic features can complement contextual multilingual transformer representations.We engineer 33 phonological, morphological, semantic, contextual, and cross-lingual features, and evaluate feature-only regressors, Solo transformer models, Hybrid transformer models, and prediction-level ensembling.Our official Closed Track submissions were generated with XLM-RoBERTa-large Solo and Hybrid models, which improved over the official baseline for all three L1 groups, achieving test RMSEs of 1.182 (ES), 1.117 (DE), and 1.006 (CN) with a mean of 1.103.We then conducted a post-submission refinement using mDeBERTa-v3-base components and a Ridge stacking ensemble, which further reduced test RMSE to 1.037 (ES), 0.997 (DE), and 0.913 (CN), with a mean of 0.982, a mean improvement of 0.121 over our best XLM-RoBERTa-large system.

Source: Workshop on Innovative Use of NLP for Building Educational Applications

DOI: 10.18653/v1/2026.bea-1.82

Authors: Adrian Pineda, Sabur Butt, Héctor Ceballos Cancino

Institution: Tecnológico de Monterrey

Language: English

Topic: Second Language Acquisition and Learning

Subfield: Developmental and Educational Psychology

Field: Psychology

Domain: Social Sciences

SDG: Quality education

Open Access status: hybrid

OpenAlex ID: W7166827226