LLM plays an 8-bit Commander X16 game using structured "smart senses"

I connected the ChatGPT API (model gpt-4o) to an 8-bit shoot-'em-up game, PvP-AI, running on a Commander X16 emulator.

Instead of pixels or audio, the model receives structured text summaries, what I’m calling "smart senses", based on the game's existing touch and EMF-style inputs.

The LLM maintains notes between turns, develops strategies across games, and even discovers an exploit in the built-in AI's behavior.

Write-up, technical details, and three sequential gameplay recordings: https://pvp-ai.russell-harper.com/#v3