Work

The Analyst in the Prompt: Role, Retrieval, and Memory Biases in LLM Financial Analysis

Year: 2026
Type: conference-paper
Abstract: Large Language Models (LLMs) use persistent user context such as memory, profiles, and role prompts.This enables personalization, but can destabilize evidence-based judgment: the same evidence may yield different conclusions under different user contexts.Finance offers a highstakes long-document setting where evidence reading is central.We test this on 3,575 SEC filings across twelve LLMs.By comparing persona-conditioned retrieval, neutral retrieval, and memory-framed context, we find that usercontext spillover comes mainly from how models interpret identical evidence under role-style context, not primarily from which evidence they retrieve.As risk-reduction techniques rather than guarantees, reframing the same mindset from assistant role to user-memory profile, and dual-output scoring, reduce-but do not eliminate-the spillover, with effectiveness that varies substantially across models.
Language: English
Open Access status: bronze
OpenAlex ID: W7208728895
FWCI:
Cites:
Cited by:
Related to: 10