Unfortunately, mere nondeterminism still admits unsatisfactorily constrained output distributions. Training these models imbues them with certain inherent attractors, some good, some bad.

We see some of these attractors across most or all frontier models due to similarities in datasets and training methods. We are seeing a digital manifestation of Jung's collective unconscious. And we have seen just how reinforcing archetypes are in human behavior over time. Sometimes it's good, sometimes it's pretty bad.

For one, it has turned us into socially programmable machines who can be manipulated quite easily by associating desired thoughts and behavior with certain archetypes: the patriarchal/matriarchal nature of US politics is a great example of this. The same evolutionary predispositions which causes children to listen and adhere to their parents is used to cause adults to listen and adhere to their politicians.

Given the nature of the core datasets these models are trained on, we should assume they are also vulnerable to such archetype-coded social manipulation, and this is troubling in a post-singularity world: When raising domesticated rats, I learned that I had to be careful what I fed them. Rats in the wild are careful about eating new substances. They will nibble and wait a while, see how it affects them or other rats, before committing to larger bites. Domesticated rats, however, will often blindly eat anything you feed them because they have learned to trust you as a safe source of food. We are going to have this exact relationship with our digital agents.