Work

How an Everyday Word in a Tool Schema Changes AI Agents' Searches While Every Call Stays Valid

Year: 2026
Type: preprint
Abstract: Can an everyday word allowed as a tool value change an AI agent's searches while every call stays valid? A traveller can call any place to stay a "hotel", but a booking tool's type field separates hotel from guesthouse. Earlier work found schema names change accuracy, and on MultiWOZ, a public booking corpus, this field has the most errors for one dialogue tracker. That work did not measure exclusion of every fitting option, test schema updates or compare repairs. When the traveller first says "hotel" and the recorded task sets no type, 5.6% to 18.8% of seven models' valid answers would have excluded every venue fitting the recorded task. In two tests registered before opening the answers, we renamed type: hotel to property_kind: large_property. Counting large_property as hotel, answers with a hotel value fell by 13.0 percentage points where the task sets no type and 29.1 points where its first or final choice is a hotel. Exploratory tool-text rewrites after a commission statement and a request to favour one type raised one agent's exclusions by about 19 points. An exploratory release test compares each model's answers under the old and new schema. With its original rule fixed before each run, it flagged the rename on all five models that received it, flagged none of 9 decisions on unchanged or reordered schemas and flagged 2 of 7 decisions on paraphrases, and neither paraphrase flag recurred. We revised the rule after seeing these results. Against corpus labels, all four planned repairs failed a preset rule for cutting extra values without losing too many needed ones (exploratory). Against the traveller's explicit words, a verifier asking the model whether the traveller excludes guesthouses passed on 5 of 9 models (post hoc), but this reference asks nearly the verifier's question. A symbolic database rule without model calls avoided about as many displays with no fitting venue as the verifier (post hoc). Builders should measure exclusion, test schema updates before release and set references before testing repairs. This evidence covers one word in one corpus.
Language: English
Open Access status: green
OpenAlex ID: W7234060997
Cites:
Cited by:
Related to: 10