Self-Healing Data Integration Pipelines via Artificial Intelligence and Human-in-the-Loop Optimization
Year: 2026
Type: conference-paper
Abstract: This paper introduces a unified framework for AIassisted data integration in Continuous Integration and Continuous Deployment (CI/CD) environments, formulated as a constrained sequential optimisation problem. Schema mapping, transformation synthesis, continuous data-quality improvement and self-healing under schema drift are jointly optimised through probabilistic AI recommendations and human-in-the-loop feedback across iterative pipeline deployments. The proposed framework models the data integration pipeline as a dynamic system evolving over successive CI/CD iterations, reflecting the interaction among data quality, computational cost, operational risk and human effort within a single optimisation formulation. A policy-driven mechanism is developed to select processing configurations under uncertainty, while schema drift is explicitly modelled to trigger adaptive self-healing actions that ensure pipeline robustness. Furthermore, human feedback is incorporated to refine both schema alignment and decision policies, enabling a systematic balance between automation and expert intervention.
Language: English
Topic: Data Quality and Management
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7225825815