Year: 2026
Type: conference-paper
Abstract: As large language models continue to advance, ensuring their trustworthiness is critical.However, inaccessible real-world ground truth labels pose a significant challenge in high-stakes domains.Recent studies have highlighted weak-to-strong generalization, where a strong model trained only on a weak model's labels surpasses the weak model in task performance.Yet, whether critical trustworthiness properties such as robustness, fairness, and privacy can generalize similarly remains an open question.This is the first work to study this question by examining if a stronger model can enhance trustworthiness when fine-tuned on a weaker model's labels, a paradigm we term weak-tostrong trustworthiness.To address this, we introduce two fundamental fine-tuning strategies that leverage trustworthiness regularization during the fine-tuning of the weak model and the weak-to-strong transfer.Our experimental evaluation on real-world datasets reveals that while some trustworthiness properties, such as fairness, adversarial robustness, and OOD robustness, show significant improvement in trustworthiness generalization when both models were regularized, others, like privacy, do not exhibit signs of weak-to-strong trustworthiness.Our results highlight the potential of weak-tostrong trustworthiness as a practical pathway for enhancing the trustworthiness of increasingly capable AI systems, even under imperfect real-world conditions.
Institutions: Harvard University, University of Vienna
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7166886059