logoalt Hacker News

disgruntledphd2today at 7:52 PM0 repliesview on HN

This study seems both wildly underpowered, chooses relatively bad methods for analysis (splitting between below average and above average, really?) and has far too many comparisions (they claim to have used bonferroni corrections where necessary, and then don't mention it again).

All in all, it's a pretty good example of modern psychological research. Bad statistics, hyped up findings and (probably) wild over-generalisation about what this tiny study means for society/the world/my research funding.

I think the biggest flaw here is around the measurement of VO2 max. So they first ask people how often they exercise (cos no-one ever lies) and then use a linear equation to map that to VO2 max. Granted, the equation has lines for age and sex, so it's not entirely useless, but if you're only going to sample 40 students then why not try to measure things a little better?

The most unintentionally hilarious thing about their methods section is that age was not normally distributed (according to a shapiro test). They sampled students, so of course it's not going to be normally distributed. Students have a well-known bias in age.

Ultimately, regardless of whether or not this finding turns out to be true (I like the idea of it, myself) this study provides absolutely no evidence for the effect.

Note: I have a PhD in psychology, but left the field a decade plus ago. I'm both horrified by this study, and also having a lot of fun poking holes in it, maybe I should try to get back into reviewing? ;)