alt Hacker NewsI get the impression that every AI lab is desperately trying to figure out how to unambiguously separate instructions from data in their token streams. The fact that they haven't managed to yet suggests to me that it's a very, very difficult problem.
Replies
simoncion • today at 2:30 PM
> I get the impression that every AI lab is desperately trying...
Of course.
I wonder how we managed way back in the day to produce systems that can handle untrusted inputs and reliably instruct a dumb-as-bricks CPU what to do based on those inputs. Must have been black magic lost to the mists of time.
➕ show 1 reply
I think what's interesting here is that they've shipped the product despite these glaring security flaws. I've noticed that in my own professional life, at some point after the pandemic people stopped caring about security as much. Issues that would have (and should have) blocked a product launch were swept under the rug.
I suspect this comes with the territory of enshittification. As an industry we're trying to wring every last dollar from every last eyeball and we've discovered that building secure systems doesn't actually move the needle very much.