Work

Inference Perf: A Benchmarking Tool for GenAI Inference

Year: 2026

Type: article

Abstract: Inference Perf is a generative AI (GenAI) inference performance benchmarking tool aimed at benchmarking and analyzing the performance of inference deployments.It is designed to be model server-agnostic, allowing for apples-to-apples comparisons across different model servers and serving stacks.It was created as a part of the inference benchmarking and metrics standardization effort in the Kubernetes wg-serving working group (Kubernetes Authors, 2024), and seeks to standardize tooling and metrics for measuring inference performance across the Kubernetes and model server communities.

Source: The Journal of Open Source Software

DOI: 10.21105/joss.10314

Authors: Ashok Chandrasekar, Sachin Varghese, Jason Kramberger, Brendan Slabe, Chen Wang, Yuan Tang

Institutions: Google (United States), Capital One (United States), IBM (United States), Red Hat (United States)

Language: English

Topic: Software System Performance and Reliability

Subfield: Computer Networks and Communications

Field: Computer Science

Domain: Physical Sciences

Open Access status: diamond

OpenAlex ID: W7204453278

FWCI:

Cites: 3

Cited by:

Related to: 10