Source KV Reuse in Multi-Target Translation with TranslateGemma 4B
Year: 2026
Type: preprint
Abstract: TitleSource KV Reuse in Multi-Target Translation with TranslateGemma 4B AuthorJohn Finder Primary categorycs.CL AbstractA decoder-only translation model typically prefills the same source again for each target language because the target instruction precedes the source. We build a reader for 4-bit TranslateGemma 4B that runs one fixed-context source pass per job and reuses its key and value tensors at all 34 layers. Each branch keeps its official target prompt and recomputes only non-source rows. A target-specific value correction in layers 12-23 acts on reduced attention outputs. On 12 English development passages translated into Japanese, French, and Traditional Chinese, the reader retained 99.22% of native mean COMET, with a worst-direction point estimate of 98.13% for French. Targeted review found factual regressions. On one Apple M1 Max, the median time for three serial branches on two 1,301-1,305-token sources fell from 8.801 to 5.648 seconds (35.82% less time) at a fixed 48 decode steps per target. Across 12 paired jobs that completed all three translations on short and medium sources, total time increased from 81.972 to 99.253 seconds (21.08% more time). The results characterize where source reuse reduced latency in this implementation and where decode overhead and translation quality limited it. AI contribution disclosureOpenAI Codex (GPT-6 based) designed and implemented experiments under human direction, analyzed the resulting artifacts, and drafted this manuscript. OpenAI GPT-6 Astra Ultra agents contributed to the earlier research. OpenAI GPT-6.1 Sol Ultra agents assisted with technical review and reproducibility materials. Codex agents also performed the arm-blind fact review described in Section 3. The metric scores and timings were computed from the recorded model runs.
Author: Finder, John
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225638818
Cites:
Cited by:
Related to: