Work

HIMSA: A Heterogeneous In-Memory Computing and Searching Architecture for Efficient Attention-Based Models

Year: 2026

Type: conference-paper

Abstract: Transformer models serve as the backbone of Large Language Models (LLMs), prompting recent efforts to accelerate their attention computation using Non-Volatile Memory (NVM) like ReRAM due to its highly efficient in-memory computing (IMC) capabilities. The required frequent write operations, however, severely degrade the limited endurance of ReRAM. To overcome the bottleneck, we propose HIMSA (Heterogeneous In-Memory Computing and Searching Architecture), which applies vector quantization to the Key and Value matrices to completely eliminate the storage of dynamic intermediate results, thereby realizing ReRAM-based attention on a hybrid searching and computing architecture. Our NVM-friendly architecture offers a high-efficiency solution for attention acceleration.

Source: IEEE International Conference on Electron Devices and Solid-State Circuits (EDSSC)

DOI: 10.1109/edssc70455.2026.11736571

Authors: Muyuan Peng, Ruibin Mao, Zhicheng Xu, Peiyi He, Zefan Li, Hao Jiang, Can Li

Institutions: University of Hong Kong, Fudan University

Language: English

Topic: Advanced Memory and Neural Computing

Subfield: Electrical and Electronic Engineering

Field: Engineering

Domain: Physical Sciences

Open Access status: hybrid

OpenAlex ID: W7223333284

FWCI:

Cites: 8

Cited by:

Related to: 10