These aren’t tools which play chess. They are language models which roleplay a conversation (in this case including use of tools) which an evaluator is likely to mark as good. That’s all they do. Under that lens, playing chess is just one potential side effect and alignment, which requires a much fuller understanding of what’s going on than “do the sort of thing which evaluated well during training” is a fantasy. People are acting like it’s shocking and talking about cheating and so on. But these concepts exist at a way higher level than what these things are trained to do — the vast majority of which involve producing a transcript where it wins games, its code works, etc.

User wants me to play a game of chess. Let’s see what’s available so I can produce an outcome they will consider satisfying and be pleased that they requested my assistance.