Work

Mixed Distillation Helps Smaller Language Models Reason Better

Year: 2024

Type: conference-paper

Abstract: As large language models (LLMs) have demonstrated impressive multiple step-by-step reasoning capabilities in recent natural language processing (NLP) reasoning tasks, many studies are interested in distilling reasoning abilities into smaller language models (SLMs) via fine-tuning.Previous distillation methods usually utilize the capabilities of LLMs to generate chain-of-thought (CoT) samples to teach SLMs.However, this distillation approach performs poorly in certain scenarios due to the limitations of CoT.In this work, we introduce a novel Mixed Distillation (MD) framework, distilling multiple step-by-step reasoning abilities into SLMs.First, we leverage LLMs to generate multiple step-by-step reasoning rationales by sampling automatically.Then, we create highquality, well-balanced mixed thought data and design a novel multi-task loss to help SLMs better learn and adaptively activate multiple stepby-step reasoning.Our extensive experiments demonstrate that MD enhances both single-path (using either CoT or PoT) and multi-path (using both CoT and PoT) reasoning abilities of SLMs during inference across reasoning tasks.Notably, a single model generated by MD exceeds the comprehensive performance of an ensemble of two individual CoT and PoT distilled models.Mistral-7B using MD can achieve remarkable improvements of 87.5%, 74.0% and 77.1% on SVAMP, GSM8K and ASDIV, respectively, outperforming the teacher model, GPT-3.5-Turbo.We hope our work provides insight into SLMs' multiple step-by-step reasoning abilities.

Source: Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP)

DOI: 10.18653/v1/2024.findings-emnlp.91