alt Hacker NewsThe real best case scenario is using LLMs to help build deterministic systems. Instead of asking an LLM to do some task that you know will be repeated, instead ask the LLM to build a program (Python script or whatever) to do the task.
Replies
If it's a one-off script/program that doesn't require additional "domain knowledge", sure. But what if you need to give as context your whole backend repository because you need to take into account a few business rules? Why give anthropic/openai access to my "secret sauce" (e.g., company private repos)?
In that case, it's way better to simply write the code yourself.
The best case scenario of LLM is transforming input into output where both are languages and accuracy doesn't matter, e.g. "rewrite this poem in pirate speech."
But that's not worth trillions of dollars...
Making systems fully deterministic ignores the entire purpose of having agents involved.
IMHO the best of both worlds option is agents working with deterministic CLIs. Where the agent does the reasoning (and text generation) but uses CLIs to carry out all of the actions (issuing refunds, unblocking accounts, or whatever).
It's possible to get very reliable and consistent work out of agents when they're using well written prompts with well designed CLIs.