Work

From Human Hands to Host Advantage? Demonstration-Seeded vs. Vanilla DQN in ViZDoom Deathmatches

Year: 2026
Type: article
Abstract: Training autonomous first-person shooter (FPS) agents from raw pixels is sample-inefficient, and small-scale multiplayer evaluations are often confounded by network roles. This paper presents a controlled case study of a deep Q-network (DQN) agent whose replay buffer is seeded with human demonstrations in ViZDoom. Agent A receives 629 human transitions; Agent B is trained with vanilla DQN. Both agents are evaluated against a random-action baseline and against each other in a custom 1v1 deathmatch across five repeated batches, with host and joiner roles swapped. After recomputing outcomes from perepisode frag counts, Agent A achieved a 68.0% win rate against the random baseline (95% CI 54.2-79.2; McNemar p = 0.006), whereas Agent B achieved 52.0% (38.5-65.2; p = 0.22) and did not significantly outperform the baseline. Contrary to the original expectation, demonstration seeding did not accelerate convergence: the vanilla agent first reached positive test scores at epoch 3, versus epoch 5 for the seeded agent, and the final plateau was nearly identical (last-10 test means 72.6 vs. 72.7). In head-to-head play, hosts won 60.0% of duels versus 19.0% for joiners, with 21.0% ties (McNemar p < 0.001), but the effect was heterogeneous: the B-host condition showed a large advantage, while the A-host condition did not. The results highlight action-space encoding, per-episode logging, and counterbalanced host/joiner designs as critical for reproducible FPS benchmarks.
Language: English
Open Access status: diamond
OpenAlex ID: W7234409016
FWCI:
Cites: 28
Cited by:
Related to: 10