Work

Cortex: Workflow-Aware Resource Pooling and Scheduling for Agentic Serving

Year: 2025

Type: preprint

Abstract: We introduce Cortex, a prototype workflow-aware serving platform designed for agentic workloads. The core principle of Cortex is stage isolation: it provisions dedicated resource pools for each distinct stage of an agentic workflow. This simple yet powerful strategy mitigates inter-stage interference in compute and memory, leading to better KV cache utilization, higher throughput, and more predictable performance. By customizing resource allocation and scheduling within each distinct stage of agentic workflows, Cortex lays the groundwork for more advanced, agent-native serving paradigms, including malleable resource management, speculative execution of workflow branches, and a shared, multi-tiered cache for "agentic state."

Source: arXiv (Cornell University)

DOI: 10.48550/arxiv.2510.14126

Authors: Pagonas, Nikos, Chung, Yeounoh, Kaffes, Kostis, Krishnamurthy, Arvind

Topic: Cloud Computing and Resource Management

Subfield: Information Systems

Field: Computer Science

Domain: Physical Sciences

Open Access status: green

OpenAlex ID: W7092284835

Cites:

Cited by:

Related to: