Work

Ferrule: Bounded Storage-Backed Weight Residency

Year: 2026
Type: preprint
Abstract: Storage-backed weight residency requires more than a small cache: reads, transfers, and device consumers can retain resources after a request or binding is logically finished. We present Ferrule, an architecture that connects source-bound materialization plans, stage-local admission, exact completion identity, and ownership through retirement. Its implemented expert path combines a persistent bounded read worker, optional next-expert lookahead, and grouped synchronous uploads without changing the synchronous forward interface. We evaluate a single-device, text-only Qwen3.5-35B-A3B-FP8 profile using BF16 execution weights and explicit F32 boundaries. Separate public-CLI acceptance checks complete 48 requests; a transfer-policy pilot completes 96 requests across eight fresh servers, including 40 measured, output-checked requests. Grouped uploads improve observed client latency and phase-qualified goodput with lookahead, but increase time to first content with blocking reads while end-to-end latency remains nearly unchanged. With two pairs per read schedule, these are conditional, descriptive benefits rather than a universal performance result. An earlier five-pair study separately isolates the effect of read scheduling. Quality gates, selected-process profiles, component faults, and HTTP cancellation provide complementary evidence about materialization and custody. The measurements bound selected logical host and cache accounts, not physical RSS, storage traffic, or universal device release; integrated five-class conservation and end-to-end fault coverage remain unverified.
Author: Lu, Huanbing
Language: English
Open Access status: green
OpenAlex ID: W7237197829
Cites:
Cited by:
Related to: 10