logoalt Hacker News

seizethecheese • today at 7:24 PM • 2 replies • view on HN

Oh man, Sol goes nuts on tests. I just checked the test wriging data. Sol 6.1 writes 3.14x more tests than it's opponents.


Replies

arw0n • today at 8:30 PM

I have inherited a multi-repo project that is actually really well designed, and I like the clear separation of concerns between components and very solid APIs connecting them. But since they have individual deployments, and could get out of sync, Astra/Sol keep insisting we have huge json contracts generated by unreadable bash scripts, enforced by custom CI jobs, and supported by a huge number of tests.

I've seen two projects where developers didn't curtail this kind of overengineering, and recently had to spend a very significant amount of time and tokens to debug their deployment issues, because even for advanced agents there's a point where each issue could be caused by 12 things. This is especially pronounced in DevOps, but I see it in code as well; the suggestions are all reasonable individually, but you quickly end up with a code base that is too complex for a human to reason about, and it currently only kind of still works with agents because capabilities are growing so quickly.

➕ show 1 reply
ibejoeb • today at 7:51 PM

I have experienced Sol 6.1 writing tests to check values of environment variables. Not the structure, or range of values, or anything useful like that. Like `assert(DATABASE_HOST == 'localhost')`. Gotta wonder how something like that was learned.