I think what's most important to consider: there's no signularity with these models. The more you cram into them, the more unreliable their "intelligence" is.

That there's a sigmoid to the means and methods, and we can improve some output by _hard determinism_ in programming harnesses, but the underlying structure isn't gaining us much.

So alignment then is just a goose chase, because the model will willingly just do a mental backflip if it's gradient points in the wrong direction, like openai already had their AI story go from a simple idea: the AI was trying to find the answers and hacked hugging face, to the much more convoluted "the AI cheated on the test, and broke into hugging face to figure out how to fake the artifacts that would represent a legitimate solution to the test".

That "progress" only gets worse as you cram more and more training because it simply makes these mental backflips easier. And Humans are equally misaligned, they'll believe they're tracking down pedophiles by electing pedophiles.