Ah man, I was curious to read your response about priors.

> If an LLM had only ever seen 1+1=2 and that was the only math they were ever exposed to, along with the numbers 0-10, could an LLM figure out that 2+2=4?

Unless you locked your kid in a room since birth with just this information, it is not the same kind of set up is it?