Why Your Chess Opponent Is Not ChatGPT: Game AI vs Language Models

By Dawid Siedlarz · · 6 min read

Play Chess free →

Short answer: A chess engine and a chatbot are different kinds of AI. Engines such as Stockfish search through possible moves with alpha-beta pruning and score positions, so they never play an illegal move and beat every human. Large language models such as ChatGPT predict the next words of text; in published tests (ChessArena 2025, Kaggle Game Arena 2025) they lose track of the board, suggest illegal moves and play at or below amateur level.

With chatbots everywhere, a common question is whether the computer opponent in a chess game is "an AI like ChatGPT". It is not, and the difference explains a lot about what each kind of AI is good at. Game AI has a seventy-year head start, works in a completely different way, and in its own domain is far stronger.

A chess program looks at the position, lists every legal move, then every reply, then every answer to that, building a tree of future positions. At the end of each branch it scores the position (material, king safety, piece activity) and works backwards, assuming both sides play their best. This is the minimax idea.

The tree grows explosively, so engines use alpha-beta pruning: as soon as one move is proven worse than an option already found, the rest of that branch is skipped. With good move ordering this lets a program search much deeper in the same time without changing the result.

Two milestones show how far this went:

  • Deep Blue (IBM, 1997) beat world champion Garry Kasparov in a six-game match under standard tournament conditions. IBM says it evaluated 200 million positions per second on 32 processors.
  • Stockfish, the strongest free engine, combines alpha-beta search with a small neural network called NNUE that scores positions. The Stockfish team introduced NNUE in August 2020, reporting a gain of more than 80 Elo in testing.

Learning instead of hand-written rules: AlphaZero

In 2017 DeepMind showed AlphaZero, described in Science in 2018. It started from random play and learned chess, shogi and Go by playing against itself, with no human games and no built-in knowledge beyond the rules. DeepMind reports that in chess it first outperformed Stockfish after just 4 hours of training.

AlphaZero is still a game AI, not a language model: a neural network trained on the game itself, guiding a search. It does not read or write text.

How language models "play": predicting text

A large language model (LLM) such as ChatGPT, Gemini or Claude is trained to continue text. When you give it a chess position in notation, it produces the move that looks most likely to come next, based on the enormous amount of text it has read, including chess books and game records. It has no built-in board, no list of legal moves and no search tree unless one is added around it.

That works surprisingly well in familiar openings, where the same moves appear in countless games. It breaks down in new positions: the model may forget where a piece stands, move through another piece, or bring back a captured one.

What the tests show

TestWhat was measuredResult
ChessArena (Liu et al., 2025)13 language models, over 800 games, move choice and puzzlesNo model beat Maia-1100, a bot that plays like a human amateur; some lost to random play
Kaggle Game Arena (Google DeepMind and Kaggle, from August 2025)Leading LLMs playing each other with no tools and no list of legal movesIllegal moves are common enough that the rules had to cover them: 3 retries, then the game is lost
"Grandmaster-level chess without search" (Ruoss et al., Google DeepMind, 2024)A 270M-parameter transformer trained on 10 million games annotated by Stockfish 16Lichess blitz Elo 2895 against humans, with no explicit search

The third row is the important nuance. A transformer, the same basic architecture as a chatbot, can play grandmaster-level chess when it is trained on chess positions with an engine's evaluations. The problem is not neural networks; it is that general chatbots were trained on text, not on billions of evaluated positions. As of February 2026, Google reported that its Gemini 3 models led the Game Arena chess leaderboard and that the models' reasoning used familiar chess ideas such as king safety and pawn structure, which shows the gap is narrowing. It is still a very different thing from an engine that checks every legal move.

Engine vs chatbot at a glance

Chess engineLanguage model
What it doesSearches moves and scores positionsPredicts the next words of text
Trained onHand-written rules, self-play or engine-evaluated positionsLarge amounts of general text
Illegal movesNever: it only chooses from generated legal movesPossible, especially in unfamiliar positions
StrengthFar above any human (Stockfish, AlphaZero)Ranges from below amateur to improving; uneven
Explaining a move in wordsNo, it outputs moves and numbersYes, this is its strength

Where language models do help chess players

The two kinds of AI complement each other. An engine tells you which move is best; a language model can explain why in plain words, answer questions about an opening's ideas, or turn an engine's evaluation into a short lesson. The sensible pattern is the one most chess sites use: let the engine calculate and use a language model only to explain, checking anything it claims about a concrete position against the engine.

The computer opponents in our games

All the computer opponents on Extensive Harmony are classic game AI: a few hundred lines of search code that run in your browser, with no chatbot and no server involved.

  • Chess uses alpha-beta search with iterative deepening. Easy looks one move ahead and plays a random legal move about a third of the time; Medium looks two half-moves ahead and then follows all captures (a quiescence search, so it does not stop in the middle of an exchange); Hard searches up to five half-moves plus captures within two seconds and keeps the deepest search that finished.
  • Tic-Tac-Toe plays perfectly: it searches the whole game tree, so the best you can do is draw. See how to win (or never lose) at tic-tac-toe.
  • Four in a Row: on Easy the computer only takes an immediate win or blocks yours and otherwise plays at random; Medium and Hard use minimax with alpha-beta pruning, 2 and 4 moves deep.
  • Checkers (2, 4 or 8 moves ahead), Reversi (1, 3 or 5) and Mancala (1, 4 or 9) use the same kind of search.

Because they only ever pick from the list of legal moves, none of them can make an illegal move. Which classic games computers have solved completely is covered in solved games explained.

Frequently asked questions

Can ChatGPT play chess?

It can play, but not well and not reliably. Language models predict text rather than calculate positions, so they lose track of the board and sometimes suggest illegal moves. In the ChessArena study (2025), none of 13 tested language models beat Maia-1100, a bot that imitates a human amateur, and in Kaggle's Game Arena a model that fails to give a legal move after four attempts loses the game.

How does a chess computer decide on a move?

Classic chess programs search: they try moves, the opponent's replies, their own answers and so on, score the resulting positions and pick the move with the best guaranteed outcome. Alpha-beta pruning skips branches that cannot change the result. Stockfish combines this alpha-beta search with a small neural network (NNUE) that scores positions.

Is AlphaZero a language model?

No. AlphaZero (DeepMind, 2017-2018) is a reinforcement-learning system that learned chess by playing against itself, starting from nothing but the rules. It uses a neural network together with a search algorithm. DeepMind reports that it first outperformed Stockfish after 4 hours of training.

Can a neural network play strong chess without searching?

Yes, if it is trained specifically for chess. A 2024 Google DeepMind study trained a 270-million-parameter transformer on 10 million games annotated by Stockfish 16 and reached a Lichess blitz rating of 2895 against humans without explicit search. General-purpose chatbots are not trained this way.

How strong is the computer in your chess game?

Our Chess uses alpha-beta search. Easy looks one move ahead and plays a random legal move about a third of the time, Medium looks two half-moves ahead plus all captures, and Hard searches up to five half-moves plus captures within two seconds. It always plays legal moves because it only ever chooses from the list of legal moves.

Sources