Adaptive Caching and Request Routing for Latency-Sensitive Serverless Applications
Year: 2026
Type: article
Abstract: Cold-start overhead and uneven placement of warm function instances can amplify request latency in serverless deployments. This paper studies a compact joint policy, AJR-ARF, combining latency-aware routing with a decayed recency-frequency warm-instance cache. At each arrival, the dispatcher estimates network, queueing, execution, and initialization components; after dispatch, a node replaces the cached function with the largest popularity-adjusted idle age. The method does not require offline training or workload forecasts. A deterministic discrete-event simulator was executed on changing-popularity synthetic requests across six nodes. Across five random seeds, the proposed policy produced 298.6 ms mean per-run 95th-percentile response latency, compared with 597.6 ms for nearest-node LRU, 392.3 ms for queue-aware LRU, and 302.8 ms for cache-aware LRU. The experiments also report cold-start frequency, SLO attainment, and sensitivity to cache capacity, traffic intensity, and popularity skew. The large reduction comes primarily from warm-state-aware routing, while adaptive eviction provides a smaller incremental change under this model.
DOI: 10.54097/nnt69h42
Authors: Sophie van Leeuwen, Daan Vermeer, Iris Jansen
Institutions: Utrecht University, Eindhoven University of Technology
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7234708594
