logoalt Hacker News

BeetleBtoday at 4:24 PM1 replyview on HN

When I was in grad school, this was the norm across the board (engineering/physics). No one wanted to reveal their secret sauce.

Things have changed since, but in my time, if a journal required source code for publication, most of the professors in my department would not have published there.


Replies

cgetoday at 5:29 PM

>Things have changed since, but in my time, if a journal required source code for publication, most of the professors in my department would not have published there.

Even when they do require it, one of the problems for journals that require source code for publication is that there is little support for making sure that code actually works reliably. Reviewers often aren't obligated to look at code packages, and when they are, might not be expected to actually get anything running; they might not even have the resources to do so. I have done reviews where I have tried to get code to run, and oftentimes I feel like the code needs work not because of any malice, but simply because making a one-time code package that works, and continues to work, for others, over time, without updates, can be quite hard, especially when odd dependencies are involved. It's also not necessarily something related to normal reviewing of scientific content, but more things like insufficient dependency pinning, accidentally hard-coded paths, environment assumptions that worked for the authors but not the reviewers, etc.

Dealing with that process might actually be something that a journal could do as part of a publication fee, the way that some journals currently do visual editing of figures.