What Does "Latency" Mean? A Claim-to-Evidence Audit of Language-Model Performance
Year: 2026
Type: preprint
Abstract: Performance numbers can reproduce without establishing the comparison attached to them. We audit published language-model inference assertions through six source-specific reconstructions and a fixed probability sample. The workstation case reproduces 576 saved statistics and 320 displayed entries, yet nine of twelve comparator elapsed-time variability values contradict a stated threshold even after display rounding. Other cases quantify a disclosed timing-window sensitivity, support a memory estimator despite a component increase, reconcile a latency headline with saved outputs, restrict a placement observation to its tested models, and identify inconsistent model attribution. The separate 150-family cohort contains 59 eligible, 67 excluded and 24 unresolved families after documented corrections; 54 assessed assertions comprise 23 supported within scope, one source conflict and 30 unresolved. These are single-author source assessments, not independently validated error rates. We supply calculation inputs, source locators and a correction record. The contribution is a reproducible account of what particular evidence supports, not physical replication, a field-wide error estimate or a new benchmark. Preprint submitted for peer review.
Author: S, Pranesh
Institution: SRM Institute of Science and Technology
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225621722
Cites:
Cited by:
Related to: