alt Hacker News100%. Anyone who is feeding unsanitized input to an LLM is doing it wrong. It'd be just like letting users craft their own SQL queries. I think the security aspect raises an interesting (if awkward) question:
How do you sanitize inputs to an LLM? Like how can you even make a secure user-facing product with this thing?
Maybe I'm lacking imagination, but it seems to me all the great "natural language interface" solutions this is supposed to enable are pretty badly hobbled by this issue.
Even your discussion makes it "sanitized input" simply doesn't exist in relation to an LLM. At best it seems like one can prefix and filter input as much as possible, monitor the results but never assume that you are done.