Heartbleed, one of the worst bugs in terms of exploitability and reach, was a bug that many engineers would be able to spot, if they were explicitly looking for it.

That's the risk of tools like Mythos/Fable/any LLM. While a human's eyes would glaze over what looks like a standard memcpy, an LLM with the right context might instantly realize the payload length was never actually verified.

And since Heartbleed existed for years, despite the full bug existing in pretty much one file, in one of the most important libraries out there, it's right to be afraid of what other obvious bugs exist and are just waiting to be found.

On the one hand, news coverage is overblown.

But scale and accessibility are absolutely a new class of problem.

In the 1960s you could pay thousands of people to watch hundreds of cameras and listen to hundreds of phone lines to monitor people, but the cost was so enormous that unless you were in East Germany or Moscow it wasn't a realistic threat model.

Now with computers we can cheaply have thousands of cameras with cheap storage that's retained forever and automatic image processing that means everyone is exposed to that kind of surveillance, which is a brand new problem.

It’s systematic

“6-12 mo to shaky profitability + ability to quickly iterate” is a business that has a good chance of surviving while “However long it takes to be fully secure” is a business that is not only rigid but needs massive up-front capital to get there and even then there’s no guarantee that the market fit is right

And after that is something we could call the “Pareto spiral”: if a company find market fit and builds an excellent product, competitors can survive at 80% of that quality. If the “100%” fails for any reason, the competitors become the new ceiling and now their competitors can survive at 80% of that (now 64%)

And only one round in, how secure could that 64% company be?

> Oookay. Is this a Mythos problem? Or a lazy/greedy/uncaring people problem?

¿Porque no los dos?

All AI risk can be described with a narrative that ends in "some human were lazy and didn't care enough", it's just which humans and how much caring was enough.

It’s still a problem we wouldn’t have without LLMs like Mythos existing. Yes, the solutions are different when you know the full context but “it’s just bad code” doesn’t mean there isn’t a problem.

The AI labs are 100% responsible for that framing

People want simple answers to complex problems. When you find out most of society is held together with duct tape, promises, and trust, and then you have tool accelerant come along like GenAI, expectations violently meet reality. GenAI simply democratized the ability to evidence, inexpensively and at scale, in a wide variety of contexts, that "The Emperor has no clothes."