Property-Based Testing Caught a Security Bug I Never Would Have Found

> We write unit tests for the happy path, maybe a few edge cases we can imagine, but what about the inputs we'd never consider? Many times we assume that LLMs are handling these scenarios by default,

Do we?

Technically a property based test caught the issue.

What I've found surprising is that the __proto__ string is a fixed set from the strings sampling set. Whereas I'd have expected the function to return random strings in the range given.

But maybe that's my biased expectation being introduced to property-based testing with random values. It also feels like a stretch to call this a property-based test, because what is the property "setters and getters that work"? Cause I expect that from all my classes.

This kind of bug would never have happened if you used a proper programming language like Rust.

It's always good to write tests with the "The Enterprise Developer from Hell" in mind: https://fsharpforfunandprofit.com/posts/property-based-testi...

> Is this exploitable? No. ... JSON.stringify knows to skip the __proto__ field. ... However, refactors to the code could ... [cause] subtle incorrectness and sharp edge cases in your code base.

So what? This line of what-if reasoning is so annoying especially when it's analysis for a language like javascript. There's no vulnerability found here and most web developers are well aware of the risky parts of the language. This is almost as bad as all the insane false positives SAST scans dump on you.

Oh I'm just waiting to get dogpiled by people who want to tell me web devs are dumber than them and couldn't possibly be competent at anything.

TL;DR: obj[key] with user-controlled key == "__proto__" is a gift that keeps on giving; buy our AI tool that will write subtle vulnerabilities like that which you yourself won’t catch in review but then it will also write some property-based tests that maybe will