Work

What a 7B model learns from synthetic agent environments: a LoRA adapter on Qwen2.5-7B-Instruct for Indian UPI tasks, and where the gain stops

Year: 2026
Type: preprint
Abstract: We fine-tuned a LoRA adapter on Qwen2.5-7B-Instruct using 2,530 generated training rows, then tested it on the public Navital Gym UPI tasks, which include the kinds it was trained on. On the 43 current public UPI tasks the untrained base solved 6 of 43 (14%, interval 7 to 27%) and the adapter, Navital Bay, solved 38 of 43 (88%, interval 76 to 95%). The two ran on the same tasks; a paired exact McNemar test gives p = 4.7e-10 (32 tasks solved only by Bay, 0 only by the base). The gain is on kinds that have training data: mandates, for which the training data has no template, stayed at 0 of 5. The gain did not carry over to GST, a different environment: 14 of 50 for the base and 11 of 50 for Bay (p = 0.51), so we found no transfer. Claude Haiku 5.5, Sonnet 5.5 and Opus 5.5 ran on the same tasks; we did not fine-tune them on these tasks; their results are in the tables beside Bay's and are not matched on settings (section 5). One training run exists, so we make no claim about which part of the data produced which result.
Language: English
Open Access status: green
OpenAlex ID: W7233577844
Cites: 1
Cited by:
Related to: