Evaluating ESCO skill classifiers
Year: 2025
Type: article
Abstract: ESCO skill classifiers, which scan job ads to identify skills at the finest level of the taxonomy, are widely used in international statistical projects and by national employment agencies. However, to our knowledge, no systematic evaluation of these classifiers has been conducted across large sets (i.e., thousands) of ESCO skills. We introduce a method for evaluating ESCO skill classifiers, addressing two key challenges: the large number of skills (up to around 14 000) and severe class imbalance. Our approach relies on matrix sampling of skills and job ads, with clustering and stratification, and uses bootstrapping to estimate standard errors for classifier comparisons. We apply the method to three classifiers using a sample of Luxembourgish IT and finance job ads. Our results indicate that the classifiers achieve acceptable accuracy despite low recall. Notably, the text matching classifier performs competitively, even against methods based on large language models.
Source: Statistical Journal of the IAOS
Language: English
Topic: AI and HR Technologies
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W4416657310