alt Hacker News> This idea is the reason why software will keep getting slower and less reliable. Because it's wrong and yet people believe it.
Not even close to being true. Prior to the advent of LLMs writing good SIMD was hard. Writing good SIMD for arbitrary tasks was _extra_ hard. Having the entire application be written with SIMD in mind was nigh impossible, from simple string parsing to data pipelining. Having arbitrary OpenCL/CUDA offloading to GPUs was a monstrous task. And even the people who were experts in these domains didn't have the time to do it in an afternoon.
But now you can. You can literally instruct your agent to write SIMD accelerated code everywhere, or to optimize down to it. And it just _does_.
> The essence of the problem is that the LLM does exactly what you tell it.
This is correct. I implore people to try writing a SaaS with an LLM, but instead of going the classic agentic project style of "design me an app", tell it to use C++ or Rust, use extensive SIMD intrinsics and inline assembly wherever possible. Just off those two instructions the difference in code quality is staggering.
I think for well defined problems, it does a really good job, but if you're starting from scratch and don't know what to ask, it can take you on a wild goose chase in an extremely sub-optimal direction and it never backtracks from that bad design.
Architectural problems are particularly difficult for it. There are many moving parts and many ways to implement so if you want certain properties like scalability and idempotent operations, then you need to have a very clear idea what to prompt.
The other day I used Claude to improve my Rendezvous hashing (with skeleton/tree) library and it managed to solve the problem. It's a pretty small library but it used up over 50k tokens and several hours to implement the optimization and it did a good job ultimately but I had written a lot of tests for it so it's not like it was implementing from scratch.
At one point it spend at least 15 minutes trying to change the code to make a test case pass to save a few meaningless percentage points of performance... After seeing it struggle with this, I stopped it and told it that it should just increase the timeout of the test case by a few milliseconds. Any engineer would have done this but Claude didn't want to take that initiative. Instead, it was starting to over-engineer the solution just to dogmatically meet that test case instead of accepting the negligible loss of performance.