Work

SHARD: A Compatibility Framework for Deploying Transformer Models on Edge NPUs

Year: 2026

Type: conference-paper

Abstract: Running inference on diverse transformer models, especially for emerging architectures such as Vision-Language Models (VLMs), is often infeasible on many edge accelerators such as Neural Processing Units (NPUs). This limitation arises because transformer execution graphs frequently violate constraints imposed by vendor-provided, black-box NPU toolchains. These constraints include restricted operator sets, rigid tensor shape and layout requirements, and tight hardware limits such as limited on-chip SRAM. Consequently, many NPUs fail to execute modern transformer models end-to-end, or they fall back to CPU execution resulting in degraded performance and efficiency.

Source: Workshop on Machine Learning and Systems (EuroMLSys)

DOI: 10.1145/3805621.3807618

Authors: Adhitya Mohan, Richard Thompson, Eric Keller, Mark Zhao

Institution: University of Colorado Boulder

Language: English

Topic: Advanced Neural Network Applications

Subfield: Computer Vision and Pattern Recognition

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

OpenAlex ID: W7158060118

FWCI:

Cites: 8

Cited by:

Related to: 10