Work

DynamixSFT: Dynamic Mixture Optimization of Instruction Tuning Collections

Year: 2026

Type: conference-paper

Abstract: As numerous instruction-tuning datasets continue to emerge, dynamically balancing and optimizing their mixtures has become a critical challenge.To address this, we propose DY-NAMIXSFT, a dynamic and automated method for instruction-tuning dataset mixture optimization.We formulate the problem as a multiarmed bandit setup and introduce a Priorscaled Boltzmann Exploration that softly anchors the updated sampling distribution to the original dataset proportions, thereby preserving the inherent diversity and coverage of the collection.Sampling probabilities are updated using a lightweight 1-Step Look-ahead Reward, reflecting how much the dataset contributes to improving the model's performance at its current state.We demonstrate that DYNAMIXSFT effectively optimizes the TÜLU-2-mixture and TÜLU-3-mixture collections across 10 benchmarks, while introducing minimal computational overhead over naive sampling.Furthermore, we provide a comprehensive analysis and visualizations to offer deeper insights into the adaptive dynamics of our method.

Source: Findings of the Association for Computational Linguistics: ACL (Findings of ACL)

DOI: 10.18653/v1/2026.findings-acl.1972