I think it should be the duty of the person asking the question to find a isomorphism to another question that the majority of people can answer.

Eg., rather than asking about the risks of A,,B,C given various probabilities etc. ask them to make bets in a highly familiar environment with a resource that has uniform marginal utility to them... so eg., "suppose you were at home and your friend does..., how much of your time on a sunday would you bet to do... "

You find that when "the very same question" is asked in highly familiar terms people get it right.

Then this investigator-academic should ask: what features of their own puzzle induce the kind of mistakes they see?

In my experience its often that people are far less socially incompetent than questioners, so put "interpretation & trust" priors on terms/presentations of questions that mean they don't parse the presentation into the problem the investigator has in mind.

People who write puzzles tend to be the most bureaucratic sort of literalists who have profoundly eccentric modes of interpretation

The problem that most people are solving is: "what do i say to make this person/question go away"

> as it is that an ability to solve them means you’re capable in ways most people aren’t.

No, because the point is that they always hinge on an arbitrary distinction to give one answer. But if you made a different arbitrary distinction you'd get a different answer. And the arbitrary distinctions are, well, arbitrary. They reflect neither truth nor capability. Just whether you can read the questioner's mind as to what arbitrary and intentionally unstated assumptions they are making.

No thanks.