I've found that one of the most useful things I can do when using LLMs is paying attention to the tool calls so that I can tell when it's giving an explanation that's obviously not based on any evidence. If it explains why a bug happens without reading the relevant files, it's obviously just a guess. If I ask about what things exist that fit certain criteria, and it doesn't search online, at best it's relying on outdated knowledge from training and at worst it's literally inventing things that don't actually exist.

In a lot of scenarios for software engineering, the cost is just wasted time without anything useful as a result, and that's already bad enough. I can't even imagine working as a lawyer and not even taking the time to validate so I don't end up reprimanded by a judge in front of my clients, but there have been so many news stories like this that obviously this is not anywhere close to a universal view...

Including for about 1809 lawyers https://damiencharlotin.com/hallucinations