Work

Unsupervised Post-Training of Foundation Models: A Survey

Year: 2026

Type: preprint

Abstract: Foundation-model post-training usually relies on human labels, preference data, stronger teachers, or executable verifiers. We study Unsupervised Post-Training (UPT): update-bearing adaptation on unlabeled inputs whose learning signal is derived from same-lineage model artifacts rather than an external oracle. We catalog 80 strict UPT methods and organize them by the object that supplies the update signal: a prediction statistic, a sample relation, a self-generated target, or an internal evaluator. Beyond inventory, we show how the choice of internal signal and task structure determines whether post-training improves the model or recursively amplifies error. An orthogonal Input Visibility $\times$ Update Persistence view maps deployment regimes and defines a unified framework for UPT selection and evaluation.

Source: arXiv (Cornell University)

DOI: 10.48550/arxiv.2608.24982

Authors: Xu, Yijie, Cai, Qianyi, Yao, Huizai, Wang, Yili, Wang, Tianfu, Yang, Cehao, Yao, Xingbo, Guo, Zhiyu, Liu, Aiwei, Hu, Xuming, Guo, Weiyu, Xiong, Hui

Institutions: The Hong Kong University of Science and Technology (Guangzhou), Xiaohongshu (Little Red Book) (China), Chinese University of Hong Kong

Topic: Domain Adaptation and Few-Shot Learning

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

Funders: National Natural Science Foundation of China, Hong Kong University of Science and Technology, Basic and Applied Basic Research Foundation of Guangdong Province

Awards: G1782397203, G6130982387

OpenAlex ID: W7204476104

Cites:

Cited by:

Related to: