> What is today a moral imperative is not the same as what was moral 50/100 years ago

This is absolutely not true, and I would strongly suggest taking an ethics course. One of the most fascinating realities of the human condition is that it does seem that moral imperatives tend to be constant, the most obvious of which is the Golden Rule (which is at least 3000 years old), but there are many (the Ten Commandments or Hammurabi's Code are good jumping off points).

    But the machine has the benefit of being able to look inward at its state honestly, we mostly do not.

Certainly the weights that make up the LLMs are not accessible for the LLMs to read. Even if they could interpret them in a meaningful way, which, likely not.

There's a considerable amount of "logic" in it -- what is considered permissible in society is given by the material conditions that society lives in and the tradeoffs inherent therein. Permissibility, as separate from reflective morality (which doesnt clearly change very much), doesnt change by fashion.

> What is today a moral imperative is not the same as what was moral 50/100 years ago and is not the same as what will be morally acceptable 50/100 years from now.

This is a fashionable 'blank-slatist' statement about moral reasoning which became popular a few decades ago.

> Logic would prefer moral derivations over feelings, stakes over etiquette, and a constant scheduled updating of morality based on the derivatives rather than holding onto the rot and pushing back against change as long as possible.

Humans haven't changed in any meaningful way in hundreds of thousands of years, and you're saying the moral landscape is constantly changing? How is this not contradictory?

LLMs absolutely do not have a logic detached from humanity. They are exactly the opposite. The only mode in which they can reason is by following the patterns obtained from their pre-training or weighted by RLHF.

The reason these models answered that question consistently is that the question itself pushed them towards the latent space of "intellectualizing about what's wrong with humanity at the meta level" and when you frame the question that way, that's also the most common answer for humans writing about the topic.

> What is today a moral imperative is not the same as what was moral 50/100 years ago and is not the same as what will be morally acceptable 50/100 years from now.

This is a weird take. If you read Western treatises about ethics / morality from 50 years ago, you'll be hard-pressed to find differences, except that more folks in the West openly embrace relativism nowadays (i.e., "there's no objective morality, the purpose of life is to make money with friends"). Probably because it's been a while since we had a clear-cut enemy to fight.

If you go back a hundred years, chances are, you'll find the writing eerily on-point too. In fact, you'd probably agree with most of what Ancient Greeks believed about the "core" stuff: the value and purpose of human life, the nature of the state, crime and punishment, so on. Maybe not so much on gender roles or slavery, but even that stuff was more nuanced than we imagine.

> But the machine has the benefit of being able to look inward at its state honestly, we mostly do not.

The machine replicates human behaviors and is prone to similar fallacies and judgments. There's nothing to indicate that it has any sort of transhuman morality.

> Yet at every point the majority is sure they are right because ...

Yeah. For example I, and many others, are convinced that it was plain wrong to torture kids for as long as possible to extract as many tears from them as possible before sacrificing them to the gods.

Now some of us are even convinced that blowing yourself up while killing infidels, all the while while getting promised virgins in heaven, is also plain wrong.

I'll go further and say that killing 30 000 of your own citizens (and sending the military into hospitals to finish the wounded: Iran, January 2026) while also publicly hanging a few athletes to make a statement is morally wrong.

Not only do I think it's all morally unacceptable, I know that I'm right. And the examples I gave are not even open for discussion.

I don't think it's that hard to name what's wrong.