alt Hacker NewsI have worked on similar problems. See e.g. [1].
The LLMs I have tested have terrible world models and intuitions for how actions change the environment. They're also not great at discerning and pursuing the right goals. They're like an infinitely patient five-year old with amazing vocabulary.
[1]: https://entropicthoughts.com/updated-llm-benchmark
(more descriptions available in earlier evaluations referenced from there)
Replies
seanmcdirmid • yesterday at 7:24 PM
You can code your prompts to read and write an external world model on the side. This is what most people do who are seriously doing games with LLMs.
mnky9800n • yesterday at 5:06 PM
we should talk. i sent you an email.
I'm going to ignore all that and tell my developers working in complicated codebases that they have to use AI. I'm sure comprehending side effects in a world building text adventure is completely different that understanding spaghetti code