Your brain is mostly just a Principal Component Analysis calculator. Take away that "math formula" and you don't have intelligence either.

The LLM weights are not intelligent. But if you give an agent a mutable memory store and allow it to iterate, it is obviously intelligent. Not massively - it's constrained by the context window - but definitely somewhat.

The confusing thing is that their language ability far outpaces their true intelligence, and humans aren't used to that. Normally those things are highly correlated, so it tricks us.