Work

SpeedLLM: An FPGA Co-design of Large Language Model Inference Accelerator

Year: 2025

Type: conference-paper

Abstract: This paper introduces SpeedLLM, a neural network accelerator designed on the Xilinx Alevo U280 platform and optimized for the Tinyllama framework to enhance edge computing performance. Key innovations include data stream parallelism, a memory reuse strategy, and Llama2 operator fusion, which collectively reduce latency and energy consumption. SpeedLLM's data pipeline architecture optimizes the read-compute-write cycle, while the memory strategy minimizes FPGA resource demands. The operator fusion boosts computational density and throughput. Results show SpeedLLM outperforms traditional Tinyllama implementations, achieving up to 4.8× faster performance and 1.18× lower energy consumption, offering improvements in edge devices.

Source: ACM International Symposium on High-Performance Parallel and Distributed Computing (HPDC)

DOI: 10.1145/3731545.3735115

Authors: Peipei Wang, Wu Guan, Liping Liang, Zhijun Wang, Hanqing Luo, Zhibin Zhang

Institutions: Beijing University of Posts and Telecommunications, Chinese Academy of Sciences, Institute of Computing Technology

Language: English

Topic: Embedded Systems Design Techniques

Subfield: Hardware and Architecture

Field: Computer Science

Domain: Physical Sciences

SDG: Industry, innovation and infrastructure

Open Access status: green

OpenAlex ID: W4414210453

FWCI:

Cites: 3

Cited by:

Related to: