alt Hacker NewsThere are some novel ideas in here worth appreciating. The classic “time scramble” is absolutely worth considering, even in top level human play.
It seems strange that even with fairly abundant datasets such as Lichess, that it is still difficult to approximate human strength with a bot.
As a “low strength human”, it’s very easy to see how traditional chess engines fail to emulate humans.
As far as actual calculation look ahead goes, humans aren’t calculating to chess engine depth. How often is a human really considering every possible move to greater than a 3 move depth?
Humans are moving with instinct. We cull the decision tree based on our evaluation of how promising a branch may be, not its objective calculated strength. This is where the chess engines break down.
Humans rely on instinct, pattern recognition, and strategy. None of this translates to a chess engine unless you are a top level player with near perfect instincts and pattern recognition.
My hunch is that for approximately human strength, the lowest ELO human makes moves completely at random, where the highest ELO human with follow the brute forced top lines. The challenge is tweaking the decision tree to match human evaluation of a branch based on skill level, and a second conditional probability of selecting a branch based on the player’s ELO.
The output of the chess engine shouldn’t be deterministic, but probabilistic. History has shown when you take a probabilistic approach to a problem, normally magic falls out. I’m yet to see a probabilistic chess engine.
Replies
There have been a bunch of studies trying to evaluate something as ostensibly 'simple' as how complex a position is. And they're all terrible. It turns out that what's complex for a human is extremely difficult to define, because we can find very simple positions extremely complex (R+B v R for instance) and some very complex positions are very simple because correct plans are 'obvious'. Similarly some quiet positions can be very complex when it's difficult to find the correct plan.
And all of this also changes with level. I'm speaking of correct plans, drawing a technically drawn ending, and so on. If a player is weaker then complex for them might simply mean when there's lots of pieces doing lots of stuff, making them more likely to hang material to a simple threat. And the aforementioned RB v R ending might be trivial for them ( on the R side) because their opponents are not going to be strong enough to create plans that can apply pressure and try to make progress. They're just going to move their extra piece around hoping you blunder something.
So getting back to engine strength - I think figuring out what's complex for a human is a very basic prerequisite, amongst several, to making an engine that plays human-like. And we're just not there yet, or seemingly even especially close.
> As a “low strength human”, it’s very easy to see how traditional chess engines fail to emulate humans.
IIRC there have been attempts to introduce human-like behavior like with Shredder which was designed to introduce human-like mistakes.
I also found the approach interesting but I've also noticed real players have personalities, i.e. do they push for open or closed positions, and they have tactical strengths and weaknesses. So the probabilities of different events needs weighting.
I also coach kids and none of their moves are random in the mathematical sense. They dont play the best moves but the moves are understandable once you hear their reasoning, it's just their reasoning doesn't properly evaluate the immediate position, future shapes and transformations.
For example looking at my chess.com profile I find basically every hung piece, and I am excellent at taking advantage of pinned piece opportunities and almost never allow an imbalance due to pinned piece, but I have blind spot when it comes to clearance sacrifices both taking advantage of them and allowing them.
So I suspect a good way to approximate a human would be to bork the evaluation based on a profile and then pick from that narrowed set of candidate moves at random. Which is more or less my understanding of how the chess.com bots work