ARCHITECT: Uncertainty-Aware Dynamic Tool Learning via Causal Intervention for Open-World Agents
Year: 2026
Type: conference-paper
Abstract: Dynamic tool generation empowers Large Language Model (LLM) agents to synthesize tools on demand, yet a critical challenge remains: 32.4% of generated tools fail on first invocation.We present Causal Tool Diagnosis (CTD), a principled framework that moves beyond blackbox reliability prediction to interpretable failure attribution.CTD constructs a Structural Causal Model (SCM) capturing how specification quality, code characteristics, and execution environment jointly determine tool outcomes.Uniquely leveraging code's intervenability, we conduct controlled sandbox experiments to estimate causal effects-an advantage unavailable in pure text generation.CTD jointly predicts confidence (Spearman rank correlation coefficient ρ=0.90) and root cause attribution (78% accuracy), with attributions directly guiding targeted repairs (+9.6% success rate over error-type classification).Our ARCHI-TECT framework, integrating CTD throughout the tool lifecycle, achieves state-of-the-art on four benchmarks including StableToolBench (+3.8%),MINT (+4.6%),T-Eval (+3.7%), and SWE-bench Lite (+2.4%), with consistent improvements across all settings.
Authors: Zhangyi Wang, Jiexiang Xu, Bingnan Yu, Zongze Li
Institutions: Nanyang Technological University, Columbia University
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W7166851671