Alignment Tuning for Large Language Models: A Data-Centric Lens on Alignment Data Pipelines
Year: 2026
Type: conference-paper
Abstract: Much of the alignment tuning literature is organized around optimization objectives, while the construction of alignment data is often treated implicitly.In this survey, we adopt a data centric perspective and reframe alignment tuning as a pipeline design problem.We decompose alignment data construction into three interacting stages, response synthesis, preference evaluation, and preference instantiation, and use this framework to organize existing alignment methods into a unified taxonomy.Through this lens, we identify recurring design tradeoffs and failure modes observed across prior alignment methods, and distill a set of high level principles that clarify how pipeline design choices influence the resulting optimization signal.Finally, we outline open challenges for alignment data pipelines, including promptlevel alignment, agentic settings, and alignment under evolving objectives.
Author: Hwanjun Song
Institution: Korea Advanced Institute of Science and Technology
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7166880837