Year: 2026
Type: conference-paper
Abstract: Distilling multi-step reasoning abilities from large language models (LLMs) into compact student models remains challenging due to noisy rationales, hallucinated supervision, and static teacher-student interactions.Existing reasoning distillation methods, including mentor-based approaches, predominantly operate in an open-loop manner, implicitly assuming uniform teacher reliability and consequently propagating erroneous intermediate reasoning.We propose GateKD, a confidence-* Corresponding author: Teerapong Panboonyuen.He conceived, designed, and led all major aspects of this work, including the research direction, methodology, implementation, experiments, analysis, and manuscript preparation.His contributions were central to the scientific rigor, technical innovation, and successful completion of the study.MAR-SAIL (Motor AI Recognition Solution Artificial Intelligence Laboratory) develops advanced AI solutions for the automotive and insurance industries.Project page available at: https://kaopanboonyuen.github.io/GateKDwhen any gating component is removed.Our results highlight that confidence-gated closedloop supervision is critical for building reliable and scalable small reasoning models.
Authors: Kasidit Sermsri, Teerapong Panboonyuen
Institution: Chulalongkorn University
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7166858395