PAKP: Predictive Asymmetric KV-Paging via Centered Bounding-Box Lookahead for Sub-6GB Edge GPUs
Year: 2026
Type: preprint
Abstract: Deploying Large Language Models (LLMs) with extended context windows (32k–64k+ tokens) on consumer-tier edge hardware (e.g., NVIDIA RTX 4050 6GB VRAM) is fundamentally bottlenecked by the linear memory growth of the Key-Value (KV) cache and severe PCIe bus bandwidth constraints. Naive host offloading degrades token generation throughput to <3.2 tok/s over PCIe Gen4 (11.81 GB/s), while static eviction rules (e.g., sliding windows) suffer catastrophic recall failure on historical needle tokens. We present PAKP (Predictive Asymmetric KV-Paging), a two-tier memory engine that pairs active GPU VRAM with a 32GB page-locked Host RAM pool. PAKP introduces:1. Centered Subspace Bounding-Box Descriptors in VRAM that eliminate key-space anisotropy and retain single-token passkey spikes across 64-token pages.2. RoPE-Aligned Cross-Layer Lookahead Prefetching leveraging residual stream continuity (cos(h_L, h_{L+1}) = 0.9842) to project Layer L+1 query access during Layer L execution.3. Dual-Stream Asynchronous DMA transfers that overlap PCIe bus latency with GPU compute. Empirically evaluated on an ASUS TUF F15 (RTX 4050 6GB VRAM, 32GB DDR5 RAM), PAKP hides 90.5% of PCIe transfer latency, reduces active KV VRAM consumption by 79.2%, maintains 100% passkey recall from evicted host memory, and sustains 27.1–39.2 tokens/sec.
Author: Akhil M
Language: English
Subfield: Hardware and Architecture
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225174378