logoalt Hacker News

thrancelast Wednesday at 12:38 AM1 replyview on HN

Side note: I wonder why it's not normalized for more papers to come with a reference implementation. Wouldn't have to be efficient, or even be easily runnable. Could be a link to a repository with a few python scripts.


Replies

mike_hearnlast Wednesday at 7:35 AM

Sometimes papers do. Like everything with academia, there's no consistency and it varies mostly by field. It's especially common in CS and less common in other fields.

The main reason people don't do it is because incentives are everything, and university/government management set bad incentives. The article points this out too. They judge academics entirely by some function of paper citations, so academics are incentivized to do the least possible work to maximize that metric. There's no positive incentive to publish more than necessary, and doing so can be risky because people might find flaws in your work by checking it. So a lot of researchers hide their raw data or code for as long as possible. They know this is wrong and will typically claim they'll publish it but there's a lot of foot dragging, and whatever gets released might not be what they used to make the paper.

In the commercial world the incentives are obviously different, but the outcomes are the same. Sometimes companies want the ideas to be used as they compliment the core business, other times the ideas need to be protected to be turned into a core business. People like to think academics and industrial research are very different but everyone is optimizing for some metric, whether they like it or not.