Recursive Alignment: Supervision and Preference Preservation
Year: 2026
Type: preprint
Abstract: Recursive alignment concerns what is preserved when one generation of AI systems supplies data, judgments, or methods for aligning the next. Two problems arise: supervision must remain reliable across capability gaps, and repeated training must retain the behaviors and preferences that supervision is intended to pro- tect. We examine these problems through recursive curation, pluralistic preference aggregation, decomposition and distillation, adversarial oversight, weak-to-strong transfer, recursive evaluators, and automated alignment research. Curation analyses show how reward improvement can coexist with concentration or unequal stakeholder influence, while pluralistic mechanisms can preserve competing preference regions under explicit conditions. Oversight experiments demonstrate improvements in assisted judgment and training, with guarantees dependent on representations, decompositions, and reliable evaluation. Across these approaches, local gains leave open whether their supporting conditions survive optimization and reuse. We compare the preserved properties, assumptions, and failure modes, and derive evaluation requirements for distributional coverage, stakeholder representation, and oversight across successor stages.
Author: Yuxuan, Li
Language: English
Subfield: Safety Research
Field: Social Sciences
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W7225104386