logoalt Hacker News

benjiroyesterday at 9:29 PM1 replyview on HN

Frankly, i feel like the people downvoting my comment, are still using older LLMs. When Opus 4.5 entered the picture, there was a noticeable improvement in the way the LLM (for me), interacted with the code base, and the issues that it was able to find.

I ran Opus on some public source code, and lets just say that the picture was less rosy for the whole "human as security".

I understand people have a aversion to LLMs but it irked me the wrong way to see the amount of downvotes on here, because people disagree with a opinion. Its starting the become like reddit. As i stated before, its still your tasks as the person working with the LLM to guide it on security practices. But as somebody now 30 years in the industry, the amount of absolute crap i have seen produced as code (and security issues), makes LLMs frankly security wizards.

Stupid example: I have yet to see LLMs not use placeholders to prevent SQL injection (despite it being trained on a lot of bad code).

The amount of code i have seen, where humans just injected variables directly into the SQL... Yea, what a surprise that SQL database content get stolen like its nothing. When doing a security audit on some public code, one of the items always found by the LLMs, yep ... SQL injectable code everywhere.

A lot of practices are easy, but anybody can overlook something in their own code base. This is where LLMs are so great. You audit with multiple LLMs and you will find points that are weak or where you forgot something, even if you code security wse.

So yea, i have no issue doing discussions but the ridiculous downvotes on what seems to come from people with no clue, is amazing. Going to take a break from here.


Replies

orwinyesterday at 10:12 PM

I must only work with genius (or rather, extremely competent seniors) who keep their codebase very clean, because that never happened to me. Even in my worst job at a bank, with idiotic product dev who couldn't read a Java trace to save their lives, security was the only thing that mattered.

But like i said, this whole discussion on LLMs since Opus is out is _great_ for my ego. At first i thought i used it wrong, then my company made weekly meeting on "how to use AI" with devs who swore by it, now i'm confident I might be a bit above average after all.

Maybe it's different for tooling/network/security devs than for product devs, but i doubt our backend are _that_ complex.