Work

Superhuman AI for two-player Catan

Year: 2026
Type: preprint
Abstract: In earlier work, we trained two-player Catan agents with a three-step recipe. A network is trained by behavior cloning on the combined moves of several teacher networks, then by reinforcement learning from self-play, and the network plays with a paired-rollout search. Here we apply the recipe again to train a second generation, with one teacher descended from the first. In a series of 100 ranked games on an online two-player ladder, the resulting agent won 74 of 89 games, 83.1% (74.0 to 89.5), against opponents in the top 4% of the more than 55,000 players ranked in the ladder's previous season. Against opponents of the same ratings, the top 0.1% of those players won 62.7% (56.0 to 69.5). The difference is 20.4 percentage points (10.1 to 30.7), on a test and reference fixed before the account's first game. With each opponent rated at their displayed rating going into the game, the difference is 25.0 percentage points (15.2 to 34.8). During the series, the account's displayed rating reached the highest rating recorded for any player in the ladder's history, and the account ranked first on the ladder. Self-play training did not make the network statistically significantly stronger without search. With search, self-play training raised the win rate by 1.7 percentage points (0.7 to 2.8), statistically significant at the 5% level.
Language: English
Open Access status: green
OpenAlex ID: W7236205850
Cites:
Cited by:
Related to: 10