ChessDS.com overview

Predicting game results with machine learning

The central idea here is that we can better represent the state of a chess game by showing result probabilities for all three results: white wins, black wins, or draw. Two machine-learning models drive the analysis on this site: a result prediction model that outputs calibrated win/draw/loss probabilities for the current position, and a move prediction model that ranks the legal moves by how likely a strong human would play each one. Both are gradient-boosted decision-tree ensembles (LightGBM) built on features from Stockfish 18 evaluations, an upstream human-imitation policy network, and a range of position- and game-level signals. A tech report describing the models is available in the blog, on Zenodo, or as a PDF.

This site owes gratitude to TWIC and Lichess, both of which were sources of PGN data. Each position is enriched with ~140 features for the result model and ~70 for the move model, including Stockfish 18 evals of the current position and its candidate moves.

Some notable facts:

• The training pool is ~464k classical chess games from TWIC in which both players are rated 2400 Elo or higher; the models are evaluated on an 82k-game holdout that is never touched during training or hyperparameter tuning.
• The site defaults to a “rating-gap-unaware” result model, which sees the average rating of the two players (capped at 2700) but not the rating gap between them. A “rating-gap-aware” variant — same architecture, additionally sees the rating gap — is also trained and reaches lower normalized entropy overall.
• Inference is sequential: the models see the move history leading to the current ply, and features from previous moves matter for predicting the current move. They will not work well on a random position presented without its preceding game history.
• Candidate moves are evaluated the same as main-line moves so long as they have ≥1% move probability according to the move prediction model.
• Two latency tiers ship from the same recipe: the fast tier evaluates a position in ~200 ms and the deep tier in ~1 s on a single CPU core. Good enough to track live games at any classical time control.
• On the 82k-game holdout, the result model reaches expected calibration error around 0.002 on 6.78M positions (when it says 70%, it's right ~70% of the time), and the move model reaches 61.9% top-1 / 88.4% top-3 — outperforming an engine-best-move baseline by ~6 pp on top-1.

Annotating moves

You'll notice annotation symbols next to moves and candidates. This is more art than science, and historically there is a ton of variation in how people do this. It's a hard problem but the shift to expected value instead of raw change in engine eval is helpful (e.g. as we easily avoid giving ? to moves that don't change anything about totally won positions). Feedback would be great, by email or social.

There's also a “precision” metric which attempts to use EV and move prediction data to score quality of play. You can read more about that in the blog section of the website.

Game PGN downloads with full result prediction values and annotations will be available soon.

Contact

Feel free to reach out! You can email albert@chessds.com, connect with the website social accounts at X or Bluesky, or find the author on LinkedIn or ORCID.