Playing chess, writing code, finding security bugs, and proving mathematical theorems all seem fairly similar to thinking and don't seem much like being tall.
The crucial distinction here is that "seem" does not at all mean the same thing as "is".
Thunder seems like the anger of the gods but it isn't. We've had chess playing programs for a long time now and despite it seeming like thinking is required for them, it isn't.
The principle you're using here isn't a scientific one but magical [1]. Abandoning empiricism and rationality is not a good way to make progress.
> writing code, finding security bugs, and proving mathematical theorems
While the second can be automated as in "this is the repo go on and look for security issues", the first one and especially the last one do not ever happen alone. Terence Tao did the math proof not chatGPT that was just used as a tool, a tool can do smart things but it's not smart