I'm a big fan of Scott's, but I haven't been happy to see how this story's failure to say what it means has in some ways only made it less relevant over the years as we've come closer to building something like the item that, on the surface, it talks about.

Not to say I think it's a bad thing at all that people have a name for this concern for giving up agency to an external system, even when that system really has your best intent in mind. You can call Claude a whispering earring and people will understand what you mean pretty well. The actual consequences you're looking out for are maybe a bit subtler than your brain atrophying, I really really never liked that detail in the story, but that's fine, people will get it. By the Death of the Author, the story's a great contribution as it is.

But it just feels like a loss that there really was a different thing that the author meant, and it got lost in translation. http://web.archive.org/web/20121007235422/http://squid314.li... clarifies it. It's quite similar to a Yudkowskyan fun theory concern (and IMO really not surprising that the story was released a couple of years after the fun theory sequence): If making the right choices and doing the right thing eventually comes down to a mechanical computation, and you've got all of your resource limitations and psychological hang-ups and such things out of the way, then there's no room left to do the right thing, and you've turned yourself into a mechanism.

I don't think that's really a possible state that any system with any trace of humanity left in it will ever enter, there's too many too large search processes in the way to fit in a finite universe. So it's a pretty abstract concern even among its peers. But it's one that's fun to talk about sometimes, and one lesson we've learned from the response to this story is that mysterious, easily misinterpreted parallels in fact do not point people in the right direction to see it.

Eh I think the writer should visit poor countries and live among locals. Mistakes and poor judgements compound, and it leads to horrific living conditions (malnutrition, nonexistent hygiene, quack medicines, short-sighted violence, etc)

In fact developed countries are built on top of these "earrings": it's called math (and by extension science in general). If the math is right, it's always right, and it has spared their lucky citizens a LOT of needless misery and wheel-reinvention.

"Thinking for yourself" is overrated if you don't have good framework. See antivaxxers and their "research"

You mean like LLM AI? Seems a little too early for that no?

Either way, what is it right about? In your mind?