logoalt Hacker News

Aurornisyesterday at 4:03 PM1 replyview on HN

I wouldn’t put much weight into this paper. This is a self-reported survey paper. They gave people surveys with a lot of questions and then tried to find correlations in the data (aka p-hacking).

Even the surveys had leading questions like “affordance of childlike wonder” from the game:

> Second, quantitative data were collected in a cross-sectional survey (N=336) of players of Super Mario Bros. and Yoshi to examine the games’ affordance of childlike wonder, overall happiness in life, and burnout risk.

There are even glaring numerical errors in the abstract that should have been caught by anyone doing any level of review or proofreading:

> First, qualitative data were collected through 41 exploratory, in-depth interviews (women: n=19, 46.3%; men: n=21, 51.2%; prefer not to disclose sex: n=11, 2.4%;

That n=11 is supposed to be n=1, if you didn’t catch it. It also doesn’t explain why the n=41 survey group separate from the 300+ survey group asked about burnout.

So I know this will generate a lot of discussion about burnout, but this is not the kind of paper to draw conclusions from. Everything about it, from the self-reported survey format to the idea itself, looks like someone started with a highly specific idea (Super Mario reduces burnout) and wanted to p-hack their way to putting it in a paper.


Replies

mrandishyesterday at 7:47 PM

> This is a self-reported survey paper.

Thank you for this post! The headline claim struck me as one that would be difficult to evidence with any scientific rigor. Reading the abstract furthered this feeling but I couldn't be bothered to read the methodology, so thanks for doing it.

> Everything about it, from the self-reported survey format to the idea itself, looks like someone started with a highly specific idea (Super Mario reduces burnout) and wanted to p-hack their way to putting it in a paper.

Indeed. Even the idea that individuals can reliably self-diagnose "burn-out" in an objective way is highly dubious.