alt Hacker NewsAt some point we have to accept that there is enough training data on "being good at thinking" and that being good at thinking generalises really well.
There doesn't need to be specific narrow training data on every conceivable task.
Replies
I agree, but also, I kind of feel like real judgement and understanding are a different thing from this "being good at thinking", and that in fact, this is the reason AIs hallucinate in the first place (A generated text can fit the pattern of a good response because its accurate, or because it's bs but has similar words and structure to other texts on the topic. Without real understanding, LLMs can't tell that the truth is important)
It's not directly related to this but I'd like to add that, in my opinion (admittedly uninformed but it makes sense to me), when they manage to, not just reduce the frequency of the LLM stating made up dates, laws etc, or made up facts, but manage to make an AI understand what the truth is and deliberately make responses accurate, we'll have real AGI that will be able to do engineering, accounting, medicine... unsupervised.
And yet I would expect that if it truly was generally good at thinking as you put it that it would be better at chess if you gave it a set of general rules/tactics and have it think through and look forward to decide its moves, but it is quite bad.