Trust before scale: running LLM-maintained knowledge libraries for domain agents in a public energy agency
Year: 2026
Type: article
Abstract: Practitioners and experts are increasingly starting to build AI “experts” over their own documents: assistants for law, technical standards or climate policy that answer with sources. In the work we reviewed most of what is published about this comes from benchmarks. Much less comes from organisations that have to sign the answers. This paper reports the first weeks of such work at a regional energy and climate agency in Croatia, where language-model agents maintain knowledge libraries on energy and climate law, climate resilience, building-design review and AI practice. Each library rests on original sources and a wiki written on top of them. Four findings stand out. First, the wiki is the cheap part: most of the effort goes into the evidence, checks and tooling that make it usable, and, in our view, that is where the value lies. Second, trust is earned on hidden, independently graded tests; on these our law agent invented no sources and outperformed a strong general model in the last two of four head-to-head tests, as graded by independent language-model readers. Third, the failures that matter are ordinary, and each now has a written control, though not all are proven yet: lost qualifiers, superseded citations, answers present but not found, false claims of absence, bloated pages, unsafe workspaces and full disks. Fourth, how strict a library must be depends on what it is for. We describe the pattern, the tests, the failures and the controls, place them against recent research, and close with recommendations for other public bodies.
Authors: Velimir, Šegon, Julije, Domac
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225389725