alt Hacker News> Copying someone else's reasoning process (as best you understand it) is still a limited form of reasoning
The LLM is not copying someone else's process. All it knows it someone else's product, and its only process is to compute the most likely next token within that.
True, and this is why I said "as best you understand it". The visible (copyable) circumstances in which someone makes a reasoning decision aren't the same as the internal decision making process, so this is just cargo-cult copying - copy the surface form without understanding the reason for it.
The perhaps surprising thing is that it works so well some of the time - specifically for math and coding.