ScrollChessMaking a chess engine play like a weaker person, not a broken one
· 9 min read · Andrii Bondar
The complaint every weak bot earns
Read the reviews of any chess app with difficulty levels and the same sentence appears: it plays like a master for fifteen moves and then gives me a rook.
That is not a bug in someone's search. It is what happens when an engine is weakened by taking away its sight. A small node budget means the engine sees shallowly — and a shallow search is not uniformly weaker, it is fine until it isn't. It plays the natural move every time, right up to the moment a three-move tactic exists that it cannot see, and then it loses a piece for nothing.
People do not play like that. A weaker player is inaccurate: they see the same ideas as a stronger one, more slowly and less completely, and they choose the second-best move more often. A person who is losing plays a series of slightly wrong moves. They do not periodically forget that queens can be captured.
The second obvious approach is worse
If blindness produces absurdity, why not add a controlled dose of it — roll a die, and sometimes play a deliberately bad move?
Because that produces moves the engine itself has already evaluated as terrible. The engine knows the rook hangs; it plays it anyway because a random number said so. That is precisely the "it gifted me a piece" feeling, arrived at deliberately instead of accidentally.
Weakness as a choice among good moves
The model that works splits weakness into two knobs that correspond to two different human failings.
Sight — the node budget — is how deep the tactics go before they become invisible. This is what makes a beginner miss a two-move combination, the way beginners genuinely do.
Slack — the interesting one — is how much worse than best a move may be and still get played. Every move the bot plays is a move the search considered reasonable. It simply is not always the best one, which is what an inaccuracy looks like from the outside: a plausible move that happens to be second best.
// Centipawns of slack: a move within this much of the best is a candidate.
var slack: Int {
switch self {
case .beginner: 250
case .casual: 120
case .club: 50
case .strong: 0 // no slack: the top level always plays what it found
}
}
The property that makes this safe is arithmetic. A move that drops a queen is roughly −900 centipawns, and 900 is not within 250 of zero, so a hanging queen is never a candidate at any level. The weak levels stay weak without ever becoming absurd — not by good taste, but because the window cannot contain the catastrophe.
The detail that makes it work: a full window at the root
There is a technical catch that is easy to miss and fatal if you do.
Ordinary alpha-beta search does not return a score for every move. Once a move fails low, the search returns a bound — "this is at most X" — and stops. That bound is enough to pick the best move, which is all a normal engine needs. But it is useless here: to choose among moves within 250 centipawns of best, you have to know how much worse the alternatives actually are, and a bound is not a value.
So the root is searched with a full window, and every root move comes back with a real score. It costs pruning at the root only — a small share of the tree — and only on the levels that use slack. The strongest level, which has no slack, does not pay it.
The candidates are then weighted so near-best moves stay likelier, with the tail of the window reachable but rare. That matters more than it sounds: human inaccuracy is mostly small and occasionally not, and a uniform pick across the window produces a bot that feels erratic rather than weak.
Then you measure it, and the measurement lies to you
A level system you have not measured is four labels. So: a harness that plays the levels against each other, and against a stronger reference search that scores every move they play.
Three times in a row, the first version of that harness produced a plausible table that was wrong. All three are worth naming, because each looked like a fact about the engine.
Mate scores contaminated the average. The reference returns ±30000 for a forced mate. Subtracting one of those from an ordinary evaluation produces a number that has nothing to do with pawns — the weakest level appeared to lose an average of 865 centipawns per move, with a single move at −27136. Clamping both evaluations to ±1000 before subtracting, the way every accuracy metric does, brought the real figure to 45. The median had been right all along, which is what medians are for.
The same game was measured twice. The levels are deterministic given a seed, and the top two have little or no slack, so two "games" between the same pair were one game played twice. It was visible in the output — the same worst move printed twice with the same position — and invisible in the summary. Six random opening plies fixed it.
Unfinished games were counted as draws. Games hit a ply limit and were recorded as drawn, which made the two weakest levels look identical: five draws out of six. Adjudicating unfinished games with the reference search — three pawns or better wins it — removed the illusion.
What the numbers actually say
With the harness fixed, thirty games per pairing:
63%Casual scores against Beginner (+10 =18 −2)
73%Club against Casual
77%Strong against Club
In Elo terms that is roughly +90, +175 and +210 between neighbours: four labels that correspond to four strengths, in the right order. Accuracy agrees — average loss per move of 49, 27, 10 and 2 centipawns — and the blunder column, moves losing 300 or more, reads 3%, 0%, 0%, 0%.
That last row is the design claim from earlier, expressed as a number that was allowed to fail. It did not.
The mistake I made with these numbers
At six games per pairing, the same bottom pair scored 58%, and I wrote it down as a finding: the two weakest levels are nearly the same opponent. At thirty games it was 63%, comfortably ordered.
The 58% was noise, and I had presented it as a fact one paragraph after warning that small samples lie. It is worth stating plainly because the failure mode is universal: a plausible table is convincing regardless of how many games are behind it, and the number of games is not printed on the table.
What survived the larger sample is smaller and duller: the gap at the bottom is about half the size of the others, and 18 of those 30 games were drawn. Whether that matters is not a question the harness can answer.
The metric that no harness can compute
Every number here is a proxy. The actual question is "would a person of this level have played that move" — and that is answered by playing the thing, at the board, and noticing when the opponent stops feeling human.
Which is also why the last piece of the model is not about move choice at all. An engine that answers a move you thought about for twenty seconds in fifty milliseconds does not read as fast; it reads as something that was never thinking. So every level has a floor under its move time — not a delay added on top, which would make the strong level artificially slow, but a minimum that only a search finishing early ever spends. With one exception: when there is only one legal move, it answers immediately. People recapture instantly, and a pause on a forced move is the most artificial pause there is.