When Local Credit Assignment Breaks: Exact Diagnostics of Additive and Pairwise Reward Projection
Year: 2026
Type: preprint
Abstract: Average reward reconstruction error is an incomplete measure of the decision quality of a factored cooperative model. We provide a small, fully reproducible diagnostic of this established issue, rather than a new multi-agent reinforcement learning algorithm. In a binary one-step game with a rare joint synergy, the exact uniform least-squares additive projection has mean squared error 2⁻ⁿ − (n + 1)4⁻ⁿ but coordination regret 1 − 3n2⁻ⁿ. Adding any set of pairwise factors reduces reconstruction error without changing the wrong maximizing action. Exhaustive evaluation at up to 16 agents confirms the formulas; 800 independent sampled fits illustrate that observing the synergistic action need not repair the population projection objective. A privileged twofold weight on the known optimal action reverses the policy while increasing uniform reconstruction error. At 16 agents, additive MSE is 1.52548 × 10⁻⁵ and regret is 0.999268. However, MSE divided by reward variance is 0.999939, exposing the apparently small absolute error as a sparse-reward averaging effect. The package supplies proofs, controlled ablations, an established matrix-game check, raw results, and tests. It does not reproduce LOMAQ or establish a new failure of neural value-decomposition algorithms.
Source: Research Square
Author: Krishna Varun Kasamneni
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7225751935