Aren't you just betting against the bitter lesson though?
Given that we have an embodied intelligence that is capable of being reasonably good at physics, it would be foolish to state categorically that AI can't do physics. I am however skeptical that LLMs can do physics. My experience is that it's really great at doing the stuff I can't be arsed to do, and is therefore very useful, but it has very poor "understanding" of physics
No, that would be a fundamental misunderstanding of the bitter lesson, which is about research bets over time. The object of comparison is the technology, the underlying substrate and fundamental architecture, and how much of it can be offloaded to computation. We're talking about the same architecture here, they're both transformers. The difference doesn't exist in a relevant way to the question.