Get started
Work

Subconscious Latent Offloading: Single-Shot System-RAM Steering Across Quantized and Cross-Scale Language Models

Year: 2026
Type: preprint
Abstract: Deploying reasoning-capable Large Language Models (LLMs) on consumer edge GPUs is fundamentally constrained by VRAM capacity and PCIe/System-RAM memory bandwidth. Standard layer-offloading engines execute offloaded CPU/RAM layers on every generated token, reducing autoregressive decoding to 2–5 tokens/s.Inspired by a modular cognitive paradigm—“The brain does the thinking; the mouth speaks it”—we introduce Subconscious Latent Offloading, an architecture where a fast, self-contained GPU model (the Active Mouth) invokes a memory-mapped System-RAM slice of middle reasoning layers L12..19 (the Subconscious Brain) strictly once per query during prompt prefill. By over-lapping this single-shot CPU pass asynchronously behind GPU prefill and hybrid symbolic-continuous reasoning cycles (Caveman Anchors + Soft-Token Superposition), effective GPU blocking wait time drops to 8–12 microseconds (0.008–0.012 ms). We evaluate two distinct operating regimes across 10 rigorous distributed systems and concurrency benchmarks (P01–P10): (1) a Zero-Calibration Same-Model Precision Bridge (Wup =I, Wdown = I, zero calibration prompts, zero trained parameters), where a crisp FP16 System-RAM slice restores middle-layer arithmetic fidelity to an aggressively low-bit quantized GPU model, jumping zero-shot accuracy from 10% (0% unbridged Caveman) to 60% and reducing perplexity from 9.05 to 5.75; and (2) a Cross-Scale Subspace Translator bridging a 1.5B GPU model (d = 1536) and a 7B System-RAM slice (d = 3584) compressed via an Intra-Layer Mixed-Precision Split (1% FP16 outliers + 99% packed INT4 bulk), reducing RAM footprint by 3.48× (2816 MB → 808 MB) and CPU latency by 2.69× (125.0 ms → 45.9 ms) at 0.9993 cosine fidelity.
DOI: 10.5281/zenodo.24073661
Language: English
Open Access status: green
OpenAlex ID: W7225611908
Cites:
Cited by:
Related to: