alt Hacker NewsI have done survey methodology research and fully agree, almost assuredly when you see questions worded in a seemingly "convoluted" way like this, the reason is that there was exhaustive research that found this wording was the best balance of reliability and validity.
There is also a lot of value in a question that works well enough, that you ask consistently over long stretches of time (or long stretches of distance). Maybe it's not perfect, but the longitudinal data would be worthless if they updated the wording every single year.
Replies
I agree – I'm sure social psychologists and psychometricians have been thinking about this since forever, probably since even the dawn of modern psychometrics. Cross-cultural and cross-language validity would likely be particularly problematic with something more detailed, especially once you get entangled with things like how anger is expressed and conceptualized, the role of positive outer expressions of affect like smiling, etc.
Agreed!
Although I'm no survey expert, the thing I'd like to bring to everyone's attention is how easy it is to not take into account people that have a degree of numeric or math illiteracy... which I guess they are the main target demographic that is included by these questions (and I can also guess that they make a worryingly large part of the demographic, because our systems are rarely inclusive).
In my experience, having met people from multiple countries during the time I've been living abroad, what I have noticed is that — in this world filled with inequality — it is a privilege to be able to have a good grasp in scientific subjects. And, for lots of different factors, people have setbacks or trauma that make it difficult to learn a subject that is either boring or painful to them.
So, yes the questions are a bit convoluted, but they help paint a mental image for probably the majority with a thing that they may be closely familiar with: stairs... Plus, it probably helps statisticians get a better signal to noise out of the questions, too.