FLINT: Efficiently Exploiting High-Bandwidth Flash for Capacity-Scalable LLM Inference Acceleration
Year: 2026
Type: article
Abstract: Large language model (LLM) inference is increasingly constrained by accelerator memory capacity rather than compute throughput. High bandwidth flash (HBF) is an emerging 3D-stacked NAND flash technology that provides multi-terabyte near-accelerator capacity, making it a promising capacity tier for storing large language model (LLM) weights. However, existing high-bandwidth flash (HBF)-based proposals face three adoption challenges: they (i) rely on coarse-grained static prefetching, (ii) expose NAND flash refresh to the accelerator-visible critical path, and (iii) assume solid-state drive (SSD)-class flash-management mechanisms that are poorly matched to a read-only HBF weight tier. Ourgoalis to design an efficient HBF substrate that addresses these three challenges. To this end, we propose FLINT (flashinferencetier), a workload-driven HBF substrate for capacity-scalable LLM inference. FLINT introduces three mechanisms: (i) a hardwareburst-buffer controller, which pipelines HBF reads with accelerator-side memory requests using existing NAND flash page/cache buffers, (ii) aphantom-plane refreshmechanism, which removes refresh from the LLM decode critical path, and (iii) aread-only flash translation layer (FTL), which replaces SSD-class support for arbitrary writes with a compact table that translates logical weight bursts to physical HBF locations. We evaluate FLINT across dense and mixture-of-experts (MoE) LLM inference models. Our evaluation shows that FLINT improves LLM decode throughput by 355×, 3.0×, and 1.17× and reduces energy consumption by 294×, 2.5×, and 1.14× compared to an SSD-equipped graphics processing unit (GPU) system, an HBM-only GPU system, and a prior hybrid HBM+HBF GPU system, respectively.
Language: English
Subfield: Hardware and Architecture
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Estimated APC paid: $2,645
OpenAlex ID: W7204952095