ChessMimic Takes on the Blitz Battlefield

In the heated arena of AI-driven chess prediction, ChessMimic is stepping up with a system designed to challenge the status quo. Comprising three small encoder-only transformers, ChessMimic focuses on predicting moves, thinking time, and game outcomes. Each model is tuned to specific Elo rating bands, promising sharper skill calibration at the expense of parameter efficiency.

Outperforming the Competition

On a significant slice of Lichess Rated Blitz games, ChessMimic has demonstrated superior accuracy in human move prediction compared to Maia-2 across all Elo bands. Its 9 million parameter model stands between Maia-3's 5M and 23M parameter models but sidesteps the complexity of Maia-3's Geometric Attention Bias.

The system doesn't stop at move prediction. ChessMimic's outcome model also considers player ratings, time control, and current clock states, reaching an impressive AUC of 0.78 out of sample. This outshines Maia-2 and even logistic regression models that factor in material, ratings, and clock time.

The Clock Model - A Mixed Bag

ChessMimic's clock model, which predicts human thinking times, presents a more complicated picture. While it delivers a usable signal, its performance isn't exactly state-of-the-art. Under ALLIE-style filters, it shows a Pearson correlation of 0.41 and Spearman's rho at 0.50, with a mean absolute error of 4.10 seconds. This is compared to ALLIE's reported correlation of 0.70. So, where's the gap? It seems concentrated in per-position bucket sharpness rather than overall calibration.

A Real Player or Just Another Pawn?

ChessMimic's public demo is available at 1e4.ai, with code and model weights accessible on GitHub. This transparency is commendable, but the question remains: Is ChessMimic a real player in the AI chess landscape or just another addition to the growing list of chess predictors?

The intersection is real. Ninety percent of the projects aren't. ChessMimic shows promise, but to truly revolutionize AI chess strategy, it needs to prove itself beyond these benchmarks. Slapping a model on a GPU rental isn't a convergence thesis. Show me the inference costs. Then we'll talk.