Playing a devil's advocate. Why do we need understanding ? To take an example i would say ~99% of the population do not understand how combustion engines or how semiconductors work, what say another 1% ?
Is the fear post-apocalyptic in nature ? We need some human priesthood to carry on tradition ? why ?
Let's assume in the next decade GPT-7 PRO Ultra is cheaply ubiquitous, inspectable, reproducible, transferable, reasonably un-constrained by any institutional interests.
What say the 1% ?
Just a thought.
I grew up in a cult. Based on my experience, I believe that the most dangerous thing a human can do is to allow someone else to do their thinking for them.
It’s not going to be ubiquitous? There hasn’t been a single frontier model where generation n costs less than generation n-1 to run. So the reasonable thing is to assume that GPT-7 will cost even more than GPT-6, and more and more of the frontier of knowledge will be locked behind a giant paywall. Participating in any field will mean ponying up to the oligarchs that own the infrastructure that runs the model.
There ought to be more to life than sitting in a pod receiving sufficient nutrients from a tube, even if there was no doubt that humanity would in this way survive until the sun expands and makes the earth uninhabitable.