About ChessDS

Predicting game results with machine learning

The central idea here is that we can better represent the state of a chess game by showing result probabilities for all three results: white wins, black wins, or draw. Two machine-learning models drive the analysis on this site: a result prediction model that outputs calibrated win/draw/loss probabilities for the current position, and a move prediction model that ranks the legal moves by how likely a strong human would play each one. Both are gradient-boosted decision-tree ensembles (LightGBM) built on features from Stockfish 18 evaluations, an upstream human-imitation policy network, and a range of position- and game-level signals. See this blog post for more details, including a link to a full technical write-up.

This site owes gratitude to TWIC and Lichess, both of which were sources of PGN data. Each position is enriched with ~140 features for the result model and ~70 for the move model, including Stockfish 18 evals of the current position and its candidate moves.

Some notable facts:

• The training pool is ~464k classical chess games from TWIC in which both players are rated 2400 Elo or higher; the models are evaluated on an 82k-game holdout that is never touched during training or hyperparameter tuning.
• The site defaults to a “rating-gap-unaware” result model, which sees the average rating of the two players (capped at 2700) but not the rating gap between them. A “rating-gap-aware” variant with the same architecture is also trained and it reaches better performance overall (but less interesting for understanding the position).
• Inference is sequential: the models see the move history leading to the current ply, and features from previous moves matter for predicting the current move. They will not work well on a random position presented without its preceding game history.
• Candidate moves are evaluated the same as main-line moves so long as they have ≥1% move probability according to the move prediction model. You will occasionally see some with lower than 1% probability, which is needed in some positions to show all viable candidates.
• Two latency tiers ship from the same recipe: the fast tier evaluates a position in ~200 ms and the deep tier in ~1 s on a single CPU core. The fast tier is used to track live games at fast time controls.
• The model is much more accurate than is possible with just evals! On the 82k-game holdout, the result model reaches expected calibration error around 0.002 on 6.78M positions (when a result has 70% probability, it happens ~70% of the time), and the move model reaches 61.9% top-1 / 88.4% top-3, outperforming an engine-best-move baseline by ~6 pp on top-1.

Annotating moves

You'll notice annotation symbols next to moves and candidates. This is a hard problem, but the shift to expected value instead of raw change in engine eval is helpful (e.g. as we easily avoid giving ? to moves that don't change anything about totally won positions). In general these use signals of how natural the move is (from a policy neural network), how strong it is, whether it sacrifices material, etc. Feedback would be great, by email or social.

There's also the “XPA” metric which leverages move predictions and result predictions to score quality of play, comparing performance to that “expected” of a ~2600 GM. You can read more about that in the XPA blog, and see how it stacks up against Elo ratings and accuracy in this follow-up post.

Game PGN downloads with full result prediction values and annotations are available at each game link. If interested in the full dataset, contact me.

Contact

Feel free to reach out! You can email albert@chessds.com, connect with the website social accounts at X or Bluesky, or find me on LinkedIn or ORCID.