alt Hacker News> Fixing unreliable agents by adding more unreliable agents.
Depending on how you combine imperfect systems, you can lower or increase error rates.
> data they have learned about the world, instructions and data under analysis.
I don't think there really is a clear distinction between these things though.
And I don't mean this in a gotcha way - these things apply to people too. Not that that means they're like people, but we have successfully built enormous systems out of fallible people, and other imperfect things.
The goal is rarely "eliminate all errors" but "have a thing that's useful, given a possibility of things going wrong".
Depending on how you combine imperfect systems, you can lower or increase error rates.
Well, in principle yes one system can compensate for another, in practice, errors tend to compound and combine in interesting ways, not go away, particularly in systems like this which are unpredictable and generate slightly different responses each time without human intervention. They are not reliable in the way we're used to considering and don't work well without supervision from careful humans at present.
I don't think there really is a clear distinction between these things though.
Yes your second point is an interesting one - the divisions between these things are arbitrarily imposed on the world, but we do have these distinctions for a reason, even if they are not completely objective.
I do think this is a significant problem specific to LLMs though (that I at least see when using them) - we don't have a way to constrain them in what knowledge they consider. This can be a bonus (they are good at connecting faraway bits of maths in interesting ways), but it can be a significant problem when asking for a summary of a data - confabulations and data from outside the context easily leak into the output of the LLM. That is undesirable behaviour and we have no reliable way to constrain it.
I'm not sure that this particular effort is going to help because it doesn't change or constrain the behaviour of the actual machines and the behaviour is the problem, not the data they are fed.