logoalt Hacker News

jakevoytko • today at 8:46 PM • 0 replies • view on HN

I gave Astra full control over the design of a software factory for a few days, as a side project. Then I switched to making it justify every change that it wanted to make. It solves problems that don't exist, has tons of random verifications that don't add value, selects tortured architectures instead of asking if it can relax its assumptions a bit, etc.

Typical convo:

Me: "Hey, I see behavior X, which seems like a bug. What is causing that?"

Astra: "That's caused by a verification failure; we should be verifying the hash of this JSON output along with a permanent audit record. Let me know if you'd like to make that change."

::ask it what caused X to happen 6 or 7 times::

Astra: Tool use was limited to 1GB but the Rust compiler needed more than that, and that caused some PROBLEMS

Me: Remove that constraint

Astra: OK