Work

Presentation Order and Structured Diagnosis in a Synthetic Payment Workflow Benchmark

Year: 2026
Type: article
Abstract: We compare three orderings of identical synthetic payment-event records across 96 cases, four local model builds, two seeds, and 2,304 calls under a fixed 256-token output limit. On 80 labeled cases, trace ordering increased exact diagnosis from 187/640 (29.22%) to 248/640 (38.75%), a 9.53-percentage-point gain. Safe abstention on 16 underdetermined cases rose from 21/128 (16.41%) to 23/128 (17.97%), satisfying the registered five-percentage-point relative margin and numerical joint gate. However, three builds never safely abstained, and structurally valid commitments increased from 50/128 to 68/128. All 803 length-terminated outputs were invalid; 47 of the net 61 additional correct outputs crossed the validity boundary. Post-hoc structural diagnostics identified fixed templates, a visible marker separating every underdetermined case, and decoy timestamps that place all distractors last under trace ordering. The protocol’s intended exhaustive two-alternative ambiguity construction was not established, although all 16 observations remained non-identifying. These results demonstrate end-to-end presentation sensitivity within the recorded suite and response budget. They do not isolate temporal reasoning, establish general ambiguity recognition, or demonstrate operational safety.
Author: Ayush Ojha
Language: English
Open Access status: diamond
OpenAlex ID: W7213898552
FWCI: 2.163
Cites: 12
Cited by: 2
Related to: 10