Year: 2026
Type: conference-paper
Abstract: We present FeatEng, a novel benchmark designed to evaluate the ability of large language models (LLMs) to perform feature engineering, a critical and knowledge-intensive task in data science.FeatEng assesses LLMs by their capacity to generate Python code that transforms raw tabular data into features that improve the performance of a downstream machine learning model.Our analysis of LLM outputs reveals that success on FeatEng often requires the application of significant world and domain knowledge, alongside complex reasoning to construct novel data representations.While focused on feature engineering, the benchmark probes a confluence of abilities indicative of an LLM's broader potential for practical, data-centric problem-solving.We demonstrate that FeatEng offers a targeted and efficient approach to assess a specific but crucial aspect of LLM capabilities relevant to real-world data science applications.
Institutions: Snowflake Inc. (United States), Polish Academy of Sciences
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7140088927