A Bayesian Analysis of Win Rates of Twilight Imperium (4th Edition) Races

A few months ago I wrote a library for running multiplayer match results through a bayesian analysis to generate ordinal skill rankings for players. Internally, rather than represent a rating as a single number, e.g. Elo ratings, there is some assumed unknowable Heisenberg uncertainty around a rating, so instead it’s represented as a gaussian/normal distribution centered on μ with a spread of σ, of where the rating probably lies. With more observations and consistent performance, this curve can be narrowed.

I made a post validating the performance of Openskill to be equivalent to Microsoft TrueSkill. Recently, I’ve played a few games of Twilight Imperium with friends, and came across this  wonderful BoardGameGeek thread with almost five thousand self-reported matches of each race and the game result. It’s all self-reported, however intuitively with this many results, any dirty data from bias should come out in the wash. Below are ordinally ranked races.

Aggregate Rankings

Rating here is calculated by μ-3σ, so the algorithm is saying there’s a 99.7% probability  that race’s rating is at least what it reports.

Since race abilities are asymmetric, and tend to perform with varying degrees of success dependent on the number of opponents, I broke these down further among 3, 4, 5, and 6 player games. This is interesting to see how each is different vs their respective mean performance. Sample size of each of these is approximately the same.

3 player

4 player

5 player

6 player

Aggregate Rankings per Experience Level

One takeaway here is Winnu performs better than expected, with a high degree of variance. The race is also vastly unpopular, and played about half as often as other races so Openskill has a harder time narrowing down on a rating. We probably can’t say that experienced players will overestimate them, though. Since the source data contains a self-described “experience” classifier, we can see how this performs with beginners versus intermediate and advanced players.

3 player

4 player

5 player

6 player

If you consider yourself advanced and draw Letnev, don’t try to be fancy. Just get capital ships.

Conclusion

While there is a pronounced gradient in skill across different races, player experience is an order of magnitude more important for predicting a winner. That said, experience isn’t something you can easily control and race selection is effectively an opening move. An unpopular opening like selecting Winnu isn’t necessarily bad if your opponents are unfamiliar with it. Selecting them in a 3 player game might throw your opponents on tilt and score a win.

This shouldn’t invalidate the golden rule: Play to have fun. In a friendly game, pick a race you think you’ll have fun playing. If you really want a Death StarWar Sun, go with Muaat and if you win, know that you did it against the odds. Imperically this data should show the races are more balanced than it may appear, which is a good thing for a game that most players may only get to play a handful of times a year.

References

• https://boardgamegeek.com/thread/1908461/races-tier-list-ranking-help-collect-data 
• https://docs.google.com/spreadsheets/d/1SE1u4d-u2OoB4gpCwDTuhNKz3wAr7pJivCLF5VoAotU/edit?usp=sharing 
• https://github.com/philihp/openskill.js/blob/399a6409a5f6282774ef14659c78333ad4cfce47/benchmark/ti4.js