Work

A First Look At Efficient And Secure On-Device LLM Inference Against KV Leakage

Year: 2024

Type: conference-paper

Abstract: Running LLMs on end devices has garnered significant attention recently due to their advantages in privacy preservation. With the advent of lightweight LLM models and specially designed GPUs, on-device LLM inference has achieved the necessary accuracy and performance metrics.

DOI: 10.1145/3691555.3696827

Authors: Huan Yang, Deyu Zhang, Yudong Zhao, Yuanchun Li, Yunxin Liu

Institutions: Central South University, Tsinghua University

Language: English

Topic: Security and Verification in Computing

Subfield: Artificial Intelligence

Field: Computer Science

Domain: Physical Sciences

Open Access status: gold

Funders: National Natural Science Foundation of China, Central South University, Universitas Brawijaya, National Outstanding Youth Science Fund Project of National Natural Science Foundation of China, National Key Research and Development Program of China

Awards: G2262187162, G4195601467, G5507408182, G7644884317, G8276902833

OpenAlex ID: W4404501756

FWCI: 1.767

Cites: 7

Cited by: 6

Related to: 10