Do copies of a learned trading policy help or harm the market they trade in?
Year: 2026
Type: article
Abstract: In a companion paper [1], we created a simulated stock exchange using a market maker and twenty-four rule-based traders, and trained a reinforcement-learning agent to trade in it. This study is looking at how the market changes when there are many copies of this agent active at the same time. The market was given 1, 2, 4, 8, or 16 copies of the agent. Each episode was tested against the same episode without the copies, using the same random price path. The results showed that in the deep market described in [1], adding 16 copies of the agent made virtually no difference. Stock prices followed the true values slightly better; the distribution of returns became slightly less fat-tailed; and the revenue generated by the copies came from the market maker instead of being taken from the other agents. When the market was shallow (the market maker held a quarter as many shares), the same 16 copies broke the market in 64% of episodes -- more often than any other type of agent (including a momentum agent which traded five times as much). Replaying the copies' recorded trades into the shallow market broke it as well, which shows that it is the trades themselves causing the breakdowns. The agent's strategy depletes the market maker's inventory because it purchases a large block of shares within a few steps -- roughly half of the amount purchased is retained by the agent. Whether or not copying a reinforcement-learning based policy will disrupt a market will depend more on the market than on the policy.
Author: Zhang, Eric
Language: English
Subfield: Finance
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W7222882066
FWCI:
Cites:
Cited by:
Related to: