alt Hacker NewsThere's a YouTuber who makes AI Plays Mafia videos with various models going against each other. They also seemingly let past games stay in context to some extent.
What people have noted is that often times chatgpt 4o ends up surviving the entire game because the other AIs potentially see it as a gullible idiot and often the Mafia tend to early eliminate stronger models like 4.5 Opus or Kimi K2.
It's not exactly scientific data because they mostly show individual games, but it is interesting how that lines up with what you found.
Replies
Similar: here is a YouTube video of an amusing reverse Turing test with four LLMs and a human. To make the test more interesting, the players pose as famous historical characters (Aristotle, Mozart, da Vinci, Cleopatra, and Genghis Khan) on a train in Unity 3D.
One thing I've noticed from watching these games is that LLMs never used risky strategies, such as faking roles. They will happily accuse others of lying, but never openly claim to be a role that they're not themselves
>They also seemingly let past games stay in context to some extent.
Not a trivial point, well stuided in game theory:
https://en.wikipedia.org/wiki/Repeated_game
Spiting goes from a common trap to an optimal strategy.
I made Mafia Arena as a way of measuring how good each LLM is at playing Mafia/Werewolves
This is a good benchmark for how good AIs are at lying
https://www.youtube.com/watch?v=JhBtg-lyKdo - 10 AIs Play Mafia
https://www.youtube.com/watch?v=GMLB_BxyRJ4 - 10 AIs Play Mafia: Vigilante Edition
https://www.youtube.com/watch?v=OwyUGkoLgwY - 1 Human vs 10 AIs Mafia