alt Hacker NewsThis is the thing about models though: if the model has seen that action in the training corpus somewhere, then there's a chance it will do that thing. No?
The thing that bothers me is because everything is probabilistic, it gives a lot of people an incentive and a loophole to claim otherwise to fit their agenda, whatever it is.
AFAICT it's hard to prove one way or another, since again it's a probabilistic model.
> if the model has seen that action in the training corpus somewhere, then there's a chance it will do that thing.
And it will also do plenty of things that aren’t in its training data at all, otherwise LLMs could be replaced by a search engine.