Why Your Chess Opponent Is Not ChatGPT: Game AI vs Language Models
Short answer: A chess engine and a chatbot are different kinds of AI. Engines such as Stockfish search through possible moves with alpha-beta pruning and score positions, so they never play an illegal move and beat every human. Large language models such as ChatGPT predict the next words of text; in published tests (ChessArena 2025, Kaggle Game Arena 2025) they lose track of the board, suggest illegal moves and play at or below amateur level.
With chatbots everywhere, a common question is whether the computer opponent in a chess game is "an AI like ChatGPT". It is not, and the difference explains a lot about what each kind of AI is good at. Game AI has a seventy-year head start, works in a completely different way, and in its own domain is far stronger.
How classic game AI works: search
A chess program looks at the position, lists every legal move, then every reply, then every answer to that, building a tree of future positions. At the end of each branch it scores the position (material, king safety, piece activity) and works backwards, assuming both sides play their best. This is the minimax idea.
The tree grows explosively, so engines use alpha-beta pruning: as soon as one move is proven worse than an option already found, the rest of that branch is skipped. With good move ordering this lets a program search much deeper in the same time without changing the result.
Two milestones show how far this went:
- Deep Blue (IBM, 1997) beat world champion Garry Kasparov in a six-game match under standard tournament conditions. IBM says it evaluated 200 million positions per second on 32 processors.
- Stockfish, the strongest free engine, combines alpha-beta search with a small neural network called NNUE that scores positions. The Stockfish team introduced NNUE in August 2020, reporting a gain of more than 80 Elo in testing.
Learning instead of hand-written rules: AlphaZero
In 2017 DeepMind showed AlphaZero, described in Science in 2018. It started from random play and learned chess, shogi and Go by playing against itself, with no human games and no built-in knowledge beyond the rules. DeepMind reports that in chess it first outperformed Stockfish after just 4 hours of training.
AlphaZero is still a game AI, not a language model: a neural network trained on the game itself, guiding a search. It does not read or write text.
How language models "play": predicting text
A large language model (LLM) such as ChatGPT, Gemini or Claude is trained to continue text. When you give it a chess position in notation, it produces the move that looks most likely to come next, based on the enormous amount of text it has read, including chess books and game records. It has no built-in board, no list of legal moves and no search tree unless one is added around it.
That works surprisingly well in familiar openings, where the same moves appear in countless games. It breaks down in new positions: the model may forget where a piece stands, move through another piece, or bring back a captured one.
What the tests show
| Test | What was measured | Result |
|---|---|---|
| ChessArena (Liu et al., 2025) | 13 language models, over 800 games, move choice and puzzles | No model beat Maia-1100, a bot that plays like a human amateur; some lost to random play |
| Kaggle Game Arena (Google DeepMind and Kaggle, from August 2025) | Leading LLMs playing each other with no tools and no list of legal moves | Illegal moves are common enough that the rules had to cover them: 3 retries, then the game is lost |
| "Grandmaster-level chess without search" (Ruoss et al., Google DeepMind, 2024) | A 270M-parameter transformer trained on 10 million games annotated by Stockfish 16 | Lichess blitz Elo 2895 against humans, with no explicit search |
The third row is the important nuance. A transformer, the same basic architecture as a chatbot, can play grandmaster-level chess when it is trained on chess positions with an engine's evaluations. The problem is not neural networks; it is that general chatbots were trained on text, not on billions of evaluated positions. As of February 2026, Google reported that its Gemini 3 models led the Game Arena chess leaderboard and that the models' reasoning used familiar chess ideas such as king safety and pawn structure, which shows the gap is narrowing. It is still a very different thing from an engine that checks every legal move.
Engine vs chatbot at a glance
| Chess engine | Language model | |
|---|---|---|
| What it does | Searches moves and scores positions | Predicts the next words of text |
| Trained on | Hand-written rules, self-play or engine-evaluated positions | Large amounts of general text |
| Illegal moves | Never: it only chooses from generated legal moves | Possible, especially in unfamiliar positions |
| Strength | Far above any human (Stockfish, AlphaZero) | Ranges from below amateur to improving; uneven |
| Explaining a move in words | No, it outputs moves and numbers | Yes, this is its strength |
Where language models do help chess players
The two kinds of AI complement each other. An engine tells you which move is best; a language model can explain why in plain words, answer questions about an opening's ideas, or turn an engine's evaluation into a short lesson. The sensible pattern is the one most chess sites use: let the engine calculate and use a language model only to explain, checking anything it claims about a concrete position against the engine.
The computer opponents in our games
All the computer opponents on Extensive Harmony are classic game AI: a few hundred lines of search code that run in your browser, with no chatbot and no server involved.
- Chess uses alpha-beta search with iterative deepening. Easy looks one move ahead and plays a random legal move about a third of the time; Medium looks two half-moves ahead and then follows all captures (a quiescence search, so it does not stop in the middle of an exchange); Hard searches up to five half-moves plus captures within two seconds and keeps the deepest search that finished.
- Tic-Tac-Toe plays perfectly: it searches the whole game tree, so the best you can do is draw. See how to win (or never lose) at tic-tac-toe.
- Four in a Row: on Easy the computer only takes an immediate win or blocks yours and otherwise plays at random; Medium and Hard use minimax with alpha-beta pruning, 2 and 4 moves deep.
- Checkers (2, 4 or 8 moves ahead), Reversi (1, 3 or 5) and Mancala (1, 4 or 9) use the same kind of search.
Because they only ever pick from the list of legal moves, none of them can make an illegal move. Which classic games computers have solved completely is covered in solved games explained.
Frequently asked questions
Can ChatGPT play chess?
It can play, but not well and not reliably. Language models predict text rather than calculate positions, so they lose track of the board and sometimes suggest illegal moves. In the ChessArena study (2025), none of 13 tested language models beat Maia-1100, a bot that imitates a human amateur, and in Kaggle's Game Arena a model that fails to give a legal move after four attempts loses the game.
How does a chess computer decide on a move?
Classic chess programs search: they try moves, the opponent's replies, their own answers and so on, score the resulting positions and pick the move with the best guaranteed outcome. Alpha-beta pruning skips branches that cannot change the result. Stockfish combines this alpha-beta search with a small neural network (NNUE) that scores positions.
Is AlphaZero a language model?
No. AlphaZero (DeepMind, 2017-2018) is a reinforcement-learning system that learned chess by playing against itself, starting from nothing but the rules. It uses a neural network together with a search algorithm. DeepMind reports that it first outperformed Stockfish after 4 hours of training.
Can a neural network play strong chess without searching?
Yes, if it is trained specifically for chess. A 2024 Google DeepMind study trained a 270-million-parameter transformer on 10 million games annotated by Stockfish 16 and reached a Lichess blitz rating of 2895 against humans without explicit search. General-purpose chatbots are not trained this way.
How strong is the computer in your chess game?
Our Chess uses alpha-beta search. Easy looks one move ahead and plays a random legal move about a third of the time, Medium looks two half-moves ahead plus all captures, and Hard searches up to five half-moves plus captures within two seconds. It always plays legal moves because it only ever chooses from the list of legal moves.
Sources
- IBM: Deep Blue (IBM Heritage)
- Stockfish: Introducing NNUE Evaluation (August 2020)
- Google DeepMind: AlphaZero: Shedding new light on chess, shogi, and Go
- Silver, D. et al. (2018). A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play. Science, 362(6419).
- Ruoss, A. et al. (2024). Amortized Planning with Large-Scale Transformers: A Case Study on Chess (first published as "Grandmaster-Level Chess Without Search"). arXiv:2402.04494.
- Liu, J. et al. (2025). ChessArena: A Chess Testbed for Evaluating Strategic Reasoning Capabilities of Large Language Models. arXiv:2509.24239.
- Kaggle: Introducing Kaggle Game Arena (August 2025)
- Google: Advancing AI benchmarking with Game Arena (February 2026)
