I've been using both sol and astra to write some python and rust, and they generate the sloppiest code I've seen in a long time. I would be embarrassed to commit anything they write. The rust code especially was terrible. Had it implement a reader/writer for a binary format and it was just littered with magic numbers, terribly named variables, very compact code visually etc. It produces working code, so I guess the GPT models are mostly for people who let AI take the wheel 100%.
I looked into Sol 6.1's performance by language recently (data from a head to head battle system built on OpenCode) and Sol 6.1 is actually has a decent win rate when it uses Rust (https://micjm.com/blog/gpt-6.1-sol-long-tail-languages ) so it probably has the same problem with any language.
You're using it wrong. If you want it producing clean code, you should be telling it to clean up and refactor the code, not just treating the first thing it spits out as the final product.
AI code uses magic numbers, but usually, if you just write something like an AGENT.md file, magic number generation doesn’t happen much, does it? Code difficulty or code problems ultimately depend on individual preference, so I can’t speak carelessly, but I think magic number related problems can basically be solved through AGENT.md or harness settings.
> magic numbers, terribly named variables, very compact code
Sounds like I'd like it. I'm really tired of claude introducing pointless constants everywhere.
Do we want LLM generated and maintained code to look similar to human generated and maintained code?
I've always used AI tools to write very constrained and small functions and modules, and then stitch it all together myself. So far. Over the last year it's become evident that many people let the AI drive 100% and there are proponents (DHH notably) that this is the way.
I'll be interested to see how compilers and LLM's compare at generating assembly over the next decade. Assembly offers enough abstraction, why do we need C? \s
What skills and prompt did you use? Did you plan implementation beforehand? Did you use test driven development?
I don't mean to come across as argumentative, it's just been my experience that the supporting docs and plan are very important.