How Hard Should a Game Be? What the Experiments Found
Short answer: there is no single right difficulty, because staying and improving pull in different directions. In two randomised experiments covering about 80,000 sessions of a maths game (Lomas and colleagues, 2013), people played longer the easier the game was. A smaller analysis in the same paper found slower learning on the easiest setting. The popular 85 percent rule (Wilson and colleagues, 2019) is a result about learning algorithms, and that paper tested no people.
Ask game designers how hard a game should be and you'll usually hear the same answer: not too hard, not too easy. Too easy is boring, too hard is frustrating, and somewhere between them sits a sweet spot. It's a satisfying idea, and I'd guess it's true for some players in some games. What's odd is how rarely it has been tested by doing the obvious thing, which is to hand different people different difficulties at random and see what they do.
When researchers did that, the curve people expected didn't show up. So the answer depends on what you're trying to get out of the game, and the evidence for each answer is thinner than the confident advice suggests.
Where the 85 percent rule comes from
The number you're most likely to have seen comes from Robert Wilson, Amitai Shenhav, Mark Straccia and Jonathan Cohen (2019) in Nature Communications. They worked out mathematically how hard training should be for learning algorithms that adjust themselves by gradient descent on tasks with two possible answers. Learning was fastest at an error rate of 15.87 percent, so about 85 percent correct. They confirmed it in simulations of a one-layer network, a two-layer network reading handwritten digits, and a model of monkeys learning to judge the direction of moving dots.
News stories turned this into "we learn best when we fail 15 percent of the time". The paper doesn't go that far. It contains no experiment with human participants. The figure depends on an assumption about noise: 85 percent for Gaussian noise, 82 for Laplacian, 75 for Cauchy. The authors say the theory still has to be generalised to tasks with more than two choices and to other learning algorithms, and they point out that an ideal Bayesian learner with perfect memory has no sweet spot at all.
A game of Four in a Row isn't a two-choice classification task. Neither is a Sudoku. The rule is a clean piece of mathematics and a reasonable hypothesis about simple perceptual skills. It isn't a setting you can copy into a game.
What happened when players were assigned a difficulty at random
The most direct test I know of is by Derek Lomas, Kishan Patel, Jodi Forlizzi and Kenneth Koedinger (2013). They used Battleship Numberline, a free game on a classroom site for grades 4 to 8, where you estimate positions on a number line to hit a target. Sessions were randomly assigned to different versions, and nobody was told the game had been made harder or easier. Engagement meant how long people chose to keep playing.
| Experiment | Sessions | What changed | What happened |
|---|---|---|---|
| 1 | 10,478 | Target size and type | Largest targets: 63% hit rate, 34% more attempts and 18% more time than the smallest (29% hit rate) |
| 2 | 69,642 | Target size up to 40% of the line, time limits, item sets, item order | Greater success went with longer play, and very easy settings did not reduce it |
The hypothesis predicted an arch, with engagement rising and then falling as the game got easier. The first experiment produced a straight line, and a curved fit added nothing (p = 0.8). The authors knew the obvious objection: their easiest condition still hit under 70 percent, so maybe they had only seen one side of the arch. The second experiment pushed much further, with targets covering up to 40 percent of the line and time limits up to 30 seconds. The arch still didn't appear. In their words, low levels of challenge "never appear to negatively affect player engagement".
A commercial test points the same way. Eva Ascarza, Oded Netzer and Julian Runge, in a working paper dated December 2023, analysed a randomised experiment in a free colour-matching mobile game, with over 300,000 players followed for 12 weeks. Players past level 20 who had played little in the previous week had a 50 percent chance of getting easier boards. Within a month the eased group had played about one more day and ten more rounds, roughly 20 percent more engagement, and they also spent more money. That's one game and one company, and it only covers experienced players who were drifting away. It says nothing about beginners or about whether anyone had a better time.
Why chess players seem to disagree
Lomas and colleagues set their result against an observational study of online chess by Abuhamdeh and Csikszentmihalyi. By their account, those players enjoyed games most against an opponent rated 262 points above them, which corresponds to about a 20 percent chance of winning, and they most enjoyed close games, whoever won. That's about as far from "the easier the better" as you can get.
Both results can be right, because the two situations differ in almost every way that matters:
- Choice. Chess players picked their opponents. Battleship players were assigned a version.
- Knowing the odds. A rating tells you how hard the game will be before it starts. Battleship players got no such signal.
- Something to win. Beating a stronger player raises your rating. Battleship offered no status to gain.
- Expertise. Rated chess players have put in many hours. Most people trying a maths game for a few minutes haven't.
- What was measured. One study asked people how much they enjoyed a game. The other timed how long they stayed. Those are different things.
Lomas and colleagues offer these as hypotheses, not findings. A reasonable reading is that seeking out difficulty is something people do once they know a game, can see the odds and have something to gain. A newcomer dropped into a hard level has none of those reasons to stay.
The catch: easy may keep you playing and teach you less
The same paper has a result the authors call troubling. They compared learning curves for 1,392 players over their first 30 attempts and found that larger, easier targets went with slower improvement than smaller ones (p = 0.02). The most engaging version wasn't the most educational one.
That finding deserves more caution than the engagement result. It covers only players who stayed for more than 30 attempts, so people who left early are missing, and the authors call it preliminary. Still, it fits the common experience that a game you can't lose stops teaching you anything. It also means the question "how hard should this be?" has no answer until you say what the game is for.
Does matching difficulty to each player fix it?
If one level can't suit everyone, the natural idea is to let the game adjust itself. Nigel Robb and Bo Zhang (2022) tested this with 221 adults recruited online, who played 15 minutes of a game about catching stars and dodging meteors. One group got constant speed, one got speed that rose level by level, and one got speed that followed their performance.
The adaptive group scored higher (median 5.05, against 4.61 for constant speed and 3.74 for rising speed), and the gap between its best and worst players was much smaller. Reported flow, enjoyment, challenge and anxiety didn't differ significantly between the three groups. The authors note that paid online questionnaires can be unreliable, and earlier studies on adaptive difficulty are mixed, so this doesn't show adaptation is useless for fun. It does show that a better-fitted game isn't automatically a more enjoyable one.
What this means when you pick a level
Put together, the studies support a modest set of conclusions. If you want to relax, or you just want someone to keep playing, there's little evidence that easy is a problem. If you want to get better at something, a setting where you sometimes fail seems to be worth the discomfort, although no study here tells you the right failure rate for a board game. If you enjoy competing and know what you're up against, a somewhat stronger opponent may be the most fun you can have.
Being told what you're in for may also matter more than designers assume. We haven't tested that here, but it follows from the chess comparison: knowing the odds changes what a loss means. Whether any of this practice carries over beyond the game is a separate and much less encouraging question, and so is whether brain training works.
What we measured in our own games
Our Four in a Row has three levels, labelled before you play, and it opens on Easy. On Easy the computer takes a win, blocks your immediate win and otherwise picks any open column. On Medium it searches two moves ahead and on Hard four. A win pays 100, 150 or 250 points, so the harder levels offer a little of the status a chess rating gives.
To see how the steps feel from below, we wrote a script that plays first as a simple beginner: it takes a win if there's one, blocks if it must, and otherwise moves at random.
| Level | Games | Beginner script wins | Computer wins | Draws |
|---|---|---|---|---|
| Easy | 2,000 | 50.0% | 46.3% | 3.7% |
| Medium | 1,000 | 0.8% | 98.8% | 0.4% |
| Hard | 300 | 0.3% | 98.7% | 1.0% |
A script isn't a person, and anyone with a plan will do better than random moves. Even so, the step from Easy to Medium is a cliff. The Battleship results suggest steep steps cost you players, though nobody has run that experiment on our game. We can't fix a cliff with one hint, but the game now suggests the middle column on your first move, which is the strongest opening there is. Our Tic-Tac-Toe shows the two extremes side by side: the easy computer takes a win but never blocks, and the unbeatable one plays perfectly, so the best you can get is a draw. If you want to test all this on yourself, play three games on each level and notice when you stop wanting one more.
Frequently asked questions
How hard should a game be?
No single difficulty suits every goal. In two randomised experiments covering about 80,000 sessions of a maths game, players stayed longer the easier it was, while a smaller analysis found they learned more slowly on the easiest setting. If you want to keep playing, easier tends to win. If you want to improve, some failure seems to help.
What is the 85 percent rule?
It is a mathematical result by Wilson and colleagues (2019). For one family of learning algorithms doing two-choice tasks, learning is fastest when training accuracy is about 85 percent. The figure changes with the noise assumed, to 82 or 75 percent, and the paper ran no experiment with people.
Does a game have to be challenging to be fun?
Not according to the largest experiments on this. Lomas and colleagues found that low challenge never reduced how long players stayed in their maths game. Chess players in another study enjoyed games against somewhat stronger opponents most, but those players had chosen their opponents and knew their ratings.
Does dynamic difficulty make games more fun?
The evidence is mixed. In a 221-person experiment by Robb and Zhang (2022), adjusting the speed to each player raised scores but made no significant difference to reported enjoyment or flow. A field test on over 300,000 mobile players found that easing difficulty for those at risk of leaving increased play, but that measures behaviour, not fun.
What difficulty should a beginner choose in Four in a Row?
Start on Easy. In our test a scripted player that only takes wins and blocks threats won about half its games on Easy and about 1 percent on Medium. Move up when Easy stops surprising you, and expect a large jump.
