Since it is fast and understand images, I wonder if it can play video games. I have a harness setup for the LLM play EA FC but even the fastest LLMs are too slow for it. I need to try this with Decisions API
How would that work? "pick which button to press"?
but the decisions are stateless... or you plan on passing the last N frames?
I think the hardest part would be coming up with the instructions/inputs to send for every non-trivial action?
Get an LLM to build it and report back! Cool idea
One of the examples on the docs page is it playing a video game. Doubt it’ll be able to run anything complex though. You’re simply trading accuracy for speed.