logoalt Hacker News

germoney • today at 3:54 PM • 1 reply • view on HN

Help me get the argument please I might be dense: is the rewrite unethical or the llm use or rust or any of the combinations?


Replies

hansvm • today at 4:19 PM

The usual argument I see goes something like:

1. There are major obvious flaws

2. Most of those are obviously LLM-induced, unless there's a new breed of human trained on just the failures of LLMs and not the successes or other relevant information

3. I therefore assume that the rest of the project is an unverified LLM psychosis without meaningful human review

That's not the worst thing in the world for all software maybe. I recently found out my apartment complex in their latest AI rewrite generates SMS OTP based purely on the timestamp and ignores passwords, so I can log in as anyone else by knowing their email and having a valid email to grab the current OTP. That's a major security failing, but how bad is it really? I can grab the last-4 of their credit card numbers, pay their rent, see how much other tenants are being screwed, and so on, and only if I know their email addresses (solvable with a tiny bit of social engineering, but let's assume that's also moderately hard). How bad is that really? On the one hand, it's terrifying, since I presume they have the same level of attention to detail with respect to payment methods and PII despite my having opted out of having them stored, but (a) that's all already been exposed via dozens of breaches and is being handled behind the scenes by my bank anyway, and (b) if we examine the immediate blast radius of the known bugs then there's approximately fuck-all an attacker can do with that information.

For a multi-threaded linker? Come the fuck on. I don't care if it's written in Rust. If you ignore all of the memory ordering intrinsics and `unsafe` then maybe it's more okay, but not having easily available UAF and other memory bugs is very different from actually implementing the correct behaviour, and for a linker where you're explicitly joining together multiple independent binary blobs and choosing what and how to execute on a machine instruction level, Rust's guarantees 100% don't save you from broken, unvalidated "business logic."

➕ show 1 reply