Load Adaptive Spline Grids for KAN MoE
Year: 2026
Type: article
Abstract: Mixture-of-Experts architecture with Kolmogorov-Arnold Network (KAN) experts, where each expert's spline grid resolution dynamically adapts based on its load, rather than using a fixed resolution across all experts. We also propose Fractional KAN-MoE, which uses a continuous, fractional grid resolution instead of discrete integer values, enabling smoother and more gradual capacity adjustments. The grid resolution for each expert is updated using a Newton-step scheduler with backtracking line search and exploration decay, this allows the model to allocate more resolution to experts handling complex input regions and less to experts handling simpler ones. Experiments on California Housing, UCI Superconductivity, and the large-scale HIGGS dataset show that this dynamic adaptive approach consistently achieves lower loss than static KAN-MoE baselines, escapes loss plateaus that static resolution cannot reach, and produces interpretable, emergent specialisation across experts, all validated across multiple random seeds.
Author: Khanna, Avichal
Institution: Manipal University Jaipur
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7166646940
FWCI:
Cites:
Cited by:
Related to: