alt Hacker NewsThis is an area where hacker news shows its weakness. We have:
1. A chart showing a very low increase (1-2 percent)
2. Nothing to control scores rising in every school in America in the last school year (due to reduction of COVID effects).
3. Scores not moving immediately after the ban, but only after the start of a new school year, which means a new cohort of students muddying the data.
Yet the data fits people's biases here (regardless whether it's right or wrong), so the celebrate it and add anecdotes and explanations why it's true.
Replies
Ok here is the crucial part of the paper:
It's a difference in differences design, using individual-level test scores and de-seasonalized data (p. 13). Their wording is:
> Y_igst is the outcome of interest for student i in grade g in school s in time period t, HighAct_s is an indicator for high pre-ban smartphone activity schools, D_t is a series of time period dummies (t = 0 indicates the first period after the ban took effect), δ_s is school fixed effects, and θ_g is grade fixed effects. In this setting, β_t are the parameters of interest, reflecting the difference in the outcome of interest between treatment and comparison schools for each period, with the period before the ban serving as the omitted category, holding grade level constant.
To me some modeling choices seem a bit heavy-handed, but I'm not an economist and could not do better.
To be fair, all those details are in the paper. And a 1-2 percent increase does not seem low to me for such a measure.
Yea, it's strange that the line didn't move quickly. I would give grace for a couple weeks to a few months, but next year? The timing feels really disconnected.
I'd go further and say its a global weakness and unbelievably destructive. The bulk of current discourse today is:
1. Read a headline/tweet/instagram.
2. Decide whether or not it fits in your worldview.
3. Move forward with the confidence that you are better informed than everyone else who agrees/disagrees with it.
You see it everywhere on all sides of all beliefs.
It didn't use to be like this. We used to read articles, we used to read common news sources, we use to not have media overrun with bad actors who know exactly what to say to get the most engagement and solidify people in their own world views.
It's all over HN and I could have hoped there'd be more willingness to say "let me consider the contents and the source before deciding if I accept it". That attitude is just lost and I don't think it will be regained and I think it's the reason we are all in a death spiral.
There's no study that's good enough for HN.
I don't think I've ever seen a science or research article posted here that didn't immediately get picked apart for this or that in the comment section. The methodology is flawed. The data is flawed. The conclusions cannot be drawn. There are confounding variables not accounted for. The sources are questionable. It's become a trope at this point. Either our commenters' standards are way too high, or all of science reporting is deeply flawed.