The agent is pretending to be a person _for a reason_

The models are trained on people being people. Once you try deviate from that the model performs worse.

A huge tell for this is how well “reasoning” works. Reasoning isn’t some alternate thinking mode, it’s just (sometimes) hidden internal monologues.

It’s easy to anthropomorphise and assume the model is intuiting, but it’s more like it’s hyping it’s self up to do the thing. That said, it’s easy to confuse “being rude to the model” with giving it more tokens to “think”.

I’d be really interested in what a non word based internal monologue could look like. Google played with this a little with the diffusion based codegen stuff. I wonder how trainable a small nonverbal conceptual package could be.