Year: 2026
Type: conference-paper
Abstract: LLM routing directs queries to a cheaper model when it suffices and to an expensive model otherwise, reducing inference cost.Existing input-based routers optimize costperformance trade-offs but provide no formal bound on how often the cheaper model fails among routed queries.We adapt a proactive conformal gate framework to LLM routing.A logistic regression gate trained on text embeddings predicts per-query safety, and Clopper-Pearson conformal calibration selects a routing threshold that guarantees the violation rate among routed queries stays below α (the violation tolerance) with probability at least 1δ (the confidence level).On two benchmarks covering math reasoning (GSM8K) and multidomain knowledge (MMLU), routing between Mixtral-8x7B and GPT-4 (a 24.5× cost difference), our method maintains the target α within the δ tolerance across a sweep from 0.05 to 0.50, while a validation-tuned baseline crosses the violation boundary on GSM8K.A feasibility analysis across all 10 Router-Bench models reveals that routability is jointly model-and task-dependent.To our knowledge, this is the first input-based LLM router with distribution-free safety guarantees.
Authors: Iqtedar Uddin, André Bauer
Institution: Illinois Institute of Technology
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7166878141