LLMs are better than people at finding intricate bugs, in part because they do not tire or get bored, but people are still much better at designing coherent systems.
That's because LLMs are incoherent by design, and need extensive guardrails to be useful for tasks that demand coherence.
I share a similar experience with bug finding; I think it also helps that LLMs can simply hold much more context at once than a human, as seeing a much larger portion of the codebase simultaneously makes it much easier to spot discrepancies.
That's because LLMs are incoherent by design, and need extensive guardrails to be useful for tasks that demand coherence.
I share a similar experience with bug finding; I think it also helps that LLMs can simply hold much more context at once than a human, as seeing a much larger portion of the codebase simultaneously makes it much easier to spot discrepancies.