alt Hacker News> Based on the lectures, textbook, and homework problems, you match the "form" of the problem with similar problems in the textbook, and apply the same algorithms to solve it.
Is the ultimate goal here to train an LLM to do well at mediocre physics homework or to train it to be a high-quality tool that can do real physics?
And yes, I'm well aware that, even in elementary school, this form-matching is a thing. It's kind of sad.
The odd thing is that we do train people to do real physics, somehow despite my cynical take. I see the education process as following heuristics that are believed to have some relationship to developing real world abilities, even if we don't know why. We all worked these problems, and now we're somehow able to do physics. In the end we don't know what turns people into physicists. Or musicians, artists, etc.
An analogy is making students write 5-paragraph essays. We don't really believe that the knowledge of how to write a 5-paragraph essay is a real-world ability, and the LLM's can write them all day long. But for some reason we believed that 5-paragraph essays were a good pedagogical tool.
But whether the same heuristics apply to teaching an LLM is anybody's guess. If they learn differently than we do, then feeding them on our learning tools isn't necessarily going to help them.