The fact that data and instructions are inherently intermixed in most LLMs.
Once either gets into the LLM layer, the LLM can't tell which is which, so one can be treated as the other.
Solutions usually involve offloading some processing to deterministic, non-AI systems which differentiate between the two (like a regular computer program (ignore reflection)), which is the opposite of a "do it all in AI" push from businesses.
alt Hacker News
The fact that data and instructions are inherently intermixed in most LLMs.
Once either gets into the LLM layer, the LLM can't tell which is which, so one can be treated as the other.
Solutions usually involve offloading some processing to deterministic, non-AI systems which differentiate between the two (like a regular computer program (ignore reflection)), which is the opposite of a "do it all in AI" push from businesses.