alt Hacker NewsWhen I first tried using IDE AI coding assistants a bit over a year ago, they would frequently fail to read files and completely hallucinate the code I asked it to analyze or edit. And even when it did successfully read my code, it would often hallucinate the API of other frameworks.
I don’t think hallucinations are a major issue with agentic coding a year later? At least, not for me.
Legal writing does have the problem that there is no such thing as a correct argument that a goal-driven loop can target. But basic things like “does this case even exist” seem solvable with research agents, and even harder problems like “does this case support the argument being made” seem conceptually very similar to questions I now ask Claude daily.
Replies
> I don’t think hallucinations are a major issue with agentic coding a year later? At least, not for me.
Are you still looking at code and agent traces to know if this is true or not?
At work, we still see hallucinations across all model families as we share bot claims for each others code
An issue I come across is the attributed failures of AI of a first time use of AI.
Non-deterministic software can behave orders of magnitude different based on how input is structured.
This factor is largely missed by bringing a deterministic mindset to trying AI.
This means the same problem solution can work for one person reliably, and not another and both are true.
Assertions that begin with a deterministic assessment using a snapshot in time of both the user and models capabilities are a new feature of this new category of software that will have to be better caught and expressed.
Yes, mechanical question of "does this exist" is trivial to solve. Whether the research is actually aligned with the user intent / the AI makes claims that are substantiated by the research at hand are much harder problems.