> If anyone can point an LLM at slow code and it automatically finds a hot loop and uses a trick it found somewhere on the 'net to vectorize it, there is little point in hiring someone with a focus on that.
LLMs are not very good at performance issues unless you walk into it with a clear idea of the likely root cause and the bigger picture regarding actual hardware and desired customer experiences.
"Please make the code go faster"
vs
"I am noticing what appears to be contention between threads under workload A, B & C, but not with workload D".
These are completely different universes of capability and outcomes.
Even if an LLM can fight its way to the answer on its own, you can achieve a specific desired result much faster and with significantly lower risk if you are genuinely an expert.
I've seen a concrete example of this recently. I profiled the client's product "the hard way" and arrived at a change to a single line of code that would eliminate a mutex issue. One of the client's developers used the LLM and wound up with a change set that touched hundreds of files, but otherwise achieved approximately the same performance fix. The other developer even had my hint that it was a single file change and couldn't figure out how to do this despite prompting a leading edge model regarding this exact possibility over and over.
Taste and aesthetics apply to absolutely everything. Not just UI/UX design. Perhaps it is even more important that we care about the things that are invisible to the customer. It is certainly easier to forget about them or treat them like they don't matter as much.
I don't know why but in Typescript when I tell an LLM to write code to check if a character is of a type, it often uses a whole Set() instead of just using a string and doing indexof. It's weird because not only should strings be faster, doing it with strings should also be the prevalent way in the dataset. Yet it uses a Set.
Not my experience at all. I pointed frontier models at some PyTorch code, told it to make it faster, and it optimized it 10x fold.
You’re probably limiting the frontier models by being specific.