alt Hacker NewsHere's a non-theistic argument:
I'm a big fan of the "veil of ignorance" philosophical thought experiment[1]. Let's say you are given the freedom to design a society and its moral code. Then you will be born into that society and subject to it. The trick is you don't know who you will born into. It's a roll of a dice. What kind of society would you design such that that seems like a winning game?
I'm fairly certain that when living in that society, you would wish to feel valued by others and that others believe you deserve a certain level of dignity and respect. Since you don't know who you will be born into, that suggests that you want a society where everyone is valued and granted dignity. That in turn extends even to people who are unfortunately not able to produce material objects with a level of skill superior to what technology can produce at some specific moment in time.
If you agree with that, then we should advocate for granting all people value and dignity "just because" and not because they happen to be better at producing bytes than an LLM. That way, even if the LLM gets better at producing those bytes, you are still valued.
That line doesn't really work, since I'm not Sam Beckett and I won't suddenly Quantum Leap into anybody's body. Why shouldn't I take an asymmetric attitude? These kind of arguments try to derive impersonal values by starting from principles of self-interest. But there's no need, because self-interest is impersonal values, in disguise. Maybe you value getting personally improved in one way or another, like getting richer or more popular or stronger or acquiring offspring, but we can play the "why" game on that until it turns into some motivating abstract impersonal goody-goody value such as "I value people".
I mean, why do you wish to feel valued by others and get respect? Is it just so that they stay out of your way and give you things? What are you going to do next, with the space and the things? This assumption, that you care what they think of you, pretty much begs the question about valuing them back. What you value is what you want to do, and the other people are involved in that, and asymmetry doesn't help get it done, except very locally (that's privacy).
Personally I often wish that all the people would go away, but I value creating knowledge, and I have to admit that I'd be crap at doing that on my own, so fine, I value people really, you can all stay.