alt Hacker NewsIsn't trying a plausible sociological intervention then measuring its effectiveness a win regardless of how effective it ended up being? Either way, we know more about what's likely to work for future interventions. It's the scientific method.
These interventions target very complex systems. It's not unexpected that we need to try a bunch of things that don't work before we find something that does, to the extent that most of the things we try are ineffective, right?
Edit: I wrote the above before reaching the end of the article. By the end, the article reaches a similar conclusion, albiet with less science and more mysticism.
Maybe that means it's a good article, leading me to reach its conclusion before taking the stance itself. On the other hand, the article's leading sections gave such a strong impression that the author thought something needs to be drastically changed about the current system — the opposite stance from its conclusion — on seemingly shakey grounds, that I almost didn't finish reading it. I'm not sure what to think, aside from please leave innocent goats alone.
Replies
I think so. But a big part of the article's complaint is that we usually don't even bother checking.
The reason the article resonated with me is because policies and interventions are often sold as "this thing is self-evidently great, and if you oppose it you're an idiot/jerk", rather than "this seems promising, so let's give it a try and scale it up if it works well". When I look at proposed measures at my local government level (for example) almost all of the supporting literature focuses on why the problem is important, rather than why the proposed interventions are likely to succeed. Being "effective" as a legislator means enacting policy, so if you want to be effective you try to keep the conversation on the first part, where it's easier to find agreement.
> Isn't trying a plausible sociological intervention then measuring its effectiveness a win regardless of how effective it ended up being?
Only if the value of whatever smidgen of knowledge you gain (assuming you do actually gain any) outweighs the much-more-than-smidgen cost of the intervention to all the people who weren't helped, and in a significant number of cases were harmed, by it.
As you can see, I don't think whatever benefits are gained actually do outweigh the costs. Which means that no, continuing to try such interventions is not a win.
The lesson we should be learning from all this is to stop trying to do so many things top down. That whenever you see some kind of large scale statistic that you think is not what it should be, the last thing you should do is concoct a large scale intervention to try to change it. Or even concoct a small scale experiment to "test" the idea, with the intention of scaling it up if it seems to work. Still less should you sacrifice a poor goat who did nothing to deserve it, just to underscore the point that your intervention won't work.
I read the whole article, and in the beginning part the only thing I felt the author wanting to change was my own mind. He is drilling in the idea that common sense interventions more often than not do not work. (maybe even almost always don't work).
His proposed change at the end is that we must always guess and check (emphasis on check) for now, and we may eventually get into a "Psychohistory" level of understanding in the future, but we are far from it now.
> Isn't trying a plausible sociological intervention then measuring its effectiveness a win regardless of how effective it ended up being
No, paying $3B for information of dubious value and which was pretty obvious ahead of time is not a win.
That has to carry with it the assumption that the response to similar interventions is long lasting over time. I'm shooting from the hip here, but I'd expect culture, life experiences, etc to play a role in how interventions would play out.
If my hunch were to price right, anything we'd learn there would have a short half live and may not hold out past, say, a generation or two.
I can say anecdotally that my grandparents would respond differently to many situations and interventions than I would. They grew up during the Great Depression and he flew in the war, those two events alone very much impacted how they viewed things for the rest of their life.
Looking at those only a generation behind me and even they seem to respond very differently than I'd expect those in my generation would.