A Feasibility-Gated Memory-Centric Architecture for Local Mixture-of-Experts Inference
Year: 2026
Type: article
Abstract: Very-large-model inference is increasingly constrained by memory capacity and data movement rather than peak arithmetic throughput. Sparse mixture-of-experts (MoE) models reduce the number of parameters activated for each token, but they do not remove the need to store a very large model and can make locality harder: expert routing changes token by token, active experts may be distributed, and an apparently efficient compute-in-memory (CIM) macro can lose its advantage once intermediate buffering, activation traffic, migration, routing, precision conversion, coherence, and security metadata are included. This paper presents the LUMA architecture methodology: start from reproducible open-weight workloads, measure the complete dataflow, build top-down workload and bottom-up hardware models, and impose an explicit feasibility gate before circuit implementation. The central hypothesis is deliberately falsifiable: memory-local execution is useful only when the reduction in expensive non-local movement remains material after all new movement and buffering introduced by the architecture are accounted for. We derive simple bandwidth lower bounds for dense and MoE inference, define the evaluation envelope, compare candidate memory topologies, and propose a month-9 architecture gate before any expensive silicon commitment. This document reports a research framework and testable architecture hypotheses; it does not claim measured LUMA silicon results.
Author: DESSALLIEN, EMMANUEL
Language: English
Subfield: Hardware and Architecture
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7204211794
FWCI:
Cites:
Cited by:
Related to: