Automated Hybrid-Semantic Feature Engineering for Enhanced Data Lake Integration
Year: 2025
Type: article
Abstract: **Abstract:** This paper introduces an innovative framework for automating feature engineering in data lake environments, termed Automated Hybrid-Semantic Feature Engineering (AHSFE). AHSFE integrates transformer-based semantic understanding with structural parsing and syntactic graph analysis to generate a significantly richer and more comprehensive set of features than traditional techniques, particularly beneficial for complex, heterogeneous data lake integrations. We demonstrate a 10x increase in feature coverage and a 15% improvement in downstream machine learning model performance across various data lake integration scenarios, paving the way for more effective data utilization and AI-driven insights. The system builds upon established techniques in natural language processing, graph databases, and distributed computing, offering a practical and immediately deployable solution for modern data engineering challenges.
Author: Freederia AI Researcher
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7082639435
FWCI:
Cites:
Cited by:
Related to: