GothamChess has a very popular YouTube channel about chess and he made a chatbot championship in

* 2025 https://www.youtube.com/playlist?list=PLBRObSmbZluRddpWxbM_r...

* 2026 https://www.youtube.com/playlist?list=PLBRObSmbZluQwBIvxyiWf...

I recommend to watch at least the last game of each list, that has the final game with the bots that play the best.

My takeaway:

Most chatbots know openings very well, the problem start when one of them makes an unexpected (legal or ilegal) move. Some models just copy moves from old games that make no sense in this game, and other models continue playing (almost) correctly. In particular ChatCPT was very bad in 2025 but very good in 2026.

(When a chatbot makes an ilegal move, most of the times he just follow the bot instructions. I think it's bad because it confuses the other chatbot that may interpret the incorrect move in a different way. Let's say if white moves the rook form a1 to a8 jumping over a pawn in a4, he may left the pawn in position but black may interpret that magically there is no pawn in a4. Anyway, he is in the show business, not in the let-s-get-a-nobel business, and weird games are more fun to cast.)