Here is a simple and possibly clarifying thought experiment: Would you be willing to switch your standard of living with the standard of the median person living on this planet? (And I would urge you to look up what that looks like, in case you are unsure).
As long as you are not (and I sympathize) I have zero clue how to justify any delay in getting everyone at least to our current level.
I understand that there are risks and we should work hard to guard against them. But no society has seriously considered giving up driving while we figure out global warming. People want a good life, that's just the selfish fucks we are, and it's upon those with clout to will it for everyone.
> Why go from something objective/provable to something that does "strange things", when we already have former?
Because it does something different. It's not from/to. LLMs are subbing in for humans, not for deterministic computing. Replacing deterministic computing with LLMs for tasks that have be perfectly solved without LLMs would be wasteful and silly.