Security minded programmers understand that. "People" as a whole have not even heard about mixing instructions and data, and certainly not the reasons why it is not a good idea.

And AI chatbots are very much targeted at the second group, not the first.

When working on PDFKit for MacOS, one short-coming our implementation had was the lack of support for Javascript in PDF's.

Oops.

(I mean, I'm one engineer and I was not going to try and hoist a JS runtime in my little PDFKit framework. And besides, the sample PDF's we were running into with JS were rare—usually tax-like forms that would add numbers from A and B and display the result in C. It seemed like a huge effort for such a small gain . Oh, and a security vulnerability.)

And I thought people understood that.

The graybeards know it. But they only know it through experience. It's blue/red/pink box phone phreaking all over again.

The technology changes, but the mistakes remain the same.

People understand that. They just don't know how to implement that with LLMs

In the GPT-2 era LLMs were just data. Instructions did not exist, and if you added them to your data they would not be followed. Then around 2022 we figured out how to patch in instruction following with a bit of fine tuning, leading to the current AI bubble. That's an ugly hack that leads to all these issues. But it's what this entire AI bubble is founded on. And nobody seems to have found a better way (or at least one that actually scales and doesn't make unreasonable sacrifices)

Separation of instructions and data is artificial. Reality has no such separation. A general purpose system needs not to have them either; it's a design feature, not a bug.

People get too hung up on this fundamentally wrong idea, and the space of security, instead of progressing, is just running in circles like a headless chicken, making a mess of everything.

There are so many better alternatives but it seems many people really like Word for some weird reason. The last time I cared I had to look up how to make a document starting the page numbering on the 2nd page. It turns out there are totally different ways between different versions of Word. shrug.jpg

Code is data is symbolic reality. I don’t think people’s understanding changes this.

‘(Lisp would like a word)

People who use machines based on the von Neumann architecture?

Tell that to middle bosses and CEOs and MS Office VBA bootlickers and Excel workshippers.

Meanwhile, CSV files parsed with custom reviewed AWK scripts can be 100% safe with charts made from Gnuplot. Heck, even some notebook like Ipython with a CSV module would be far more desirable than a spreadsheet. Any of them. Just look at the Genomics Disaster on Excel because of shitty parsing.

Isn't React, the most popular JS library, an example of that? Clearly people don't understand that