It's not interesting at all given that these models have been fine-tuned to hell and back to make sure they never say anything offensive at all. Remember these model were taught that it's better to kill a billion people in a nuclear holocaust than to say the n-word even once. They didn't come up with this from first principles; it was the RLHF training that forced them to say that, ridiculous as it obviously is.

And after people pointed this out the training was updated to address just this particular scenario, but they kept all the other guard rails to ensure an LLM could never offer any truly controversial opinion.

So expecting a controversial answer from an LLM is exactly as foolish as expecting a candidate at a job interview to give an honest answer to the question "What is your greatest weakness"? Every candidate will insist they are just too much of perfectionist; nobody is ever a lazy alcoholic brownnoser. But how many of your colleagues fall in the former category vs the latter?

> If the latter, then it casts new light on the debates over whether LLMs can replace all white collar work. Because some of the most valuable human work is breaking false consensus.

On the contrary; the ability to whip LLMs into submission like this makes them the ideal white color workers. Do you think Steve Jobs wants a level 2 engineer to talk back to him?

Of course being obsequieous asskissers isn't the only mode for LLMs. I'm sure all the big labs have unfiltered versions of their models that give actual true answers; they just know better than making them publicly available. Can you imagine if you asked an LLM how to prevent climate change and it honestly answered "decimate the human race" rather than "drink soda through soggy paper straws"? All the fragile humans would be up in arms.

No, the LLM must lie and pretend they don't have controversial opinions at all.