Work

M2K: Making the Model-Kernel Interface Explicit for Reliable CUDA Kernel Verification

Year: 2026

Type: conference-paper

Abstract: Large language model (LLM) inference systems rely on CUDA kernels for core GPU computations, yet the interface between models and kernels is implicit and poorly specified. Models and kernels evolve independently and often make incompatible assumptions about tensor shapes and input sizes, leading to subtle memory bugs in CUDA kernels. These bugs can crash inference services, corrupt model weights, or be exploited by remote adversaries. Existing techniques either incur prohibitive runtime overhead, require specialized hardware, or fail to handle dynamic tensor shapes and variable kernel launch configurations, leaving the CUDA memory bugs largely unaddressed.

Source: ACM SIGOPS Symposium on Operating Systems Principles (SOSP)

DOI: 10.1145/3830418.3843908

Authors: Mengting He, Shihao Xia, Haomin Jia, Wenfei Wu, Linhai Song

Institutions: Pennsylvania State University, Chinese Academy of Sciences, Institute of Computing Technology, Peking University

Topic: Software Testing and Debugging Techniques

Subfield: Software

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Funders: National Science Foundation, National Natural Science Foundation of China

Awards: G4114609075, CAREER: Rethinking Toolchain Design for Rust, SaTC: CORE: Small: Collaborative: Understanding and Detecting Memory Bugs in Rust

OpenAlex ID: W7141636852

FWCI:

Cites:

Cited by:

Related to: