Anthropic's introspection experiments have seemed to show that your argument is falsifiable.
https://www.anthropic.com/research/introspection
> In fact, most of the time models fail to demonstrate introspection—they’re either unaware of their internal states or unable to report on them coherently.
You got the wrong takeaway from your link.
> In fact, most of the time models fail to demonstrate introspection—they’re either unaware of their internal states or unable to report on them coherently.
You got the wrong takeaway from your link.