Towards End-to-End Optimization of LLM-based Applications with Ayo
Year: 2025
Type: conference-paper
Abstract: Large language model (LLM)-based applications consist of both LLM and non-LLM components, each contributing to the end-to-end latency. Despite great efforts to optimize LLM inference, end-to-end workflow optimization has been overlooked. Existing frameworks employ coarse-grained orchestration with task modules, which confines optimizations to within each module and yields suboptimal scheduling decisions.
Authors: Xin Tan, Yimin Jiang, Yitao Yang, Hong Xu
Institution: Chinese University of Hong Kong
Language: English
Subfield: Hardware and Architecture
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W4408901326