alt Hacker NewsRatchets in software development (2021)
Comments
I’ve never understood why linters don’t have this baked in. You want to deprecate a pattern, but marking it as an error and failing the build won’t work. So you mark it warning and fill everyone’s editors with yellow lines. And then we just get used to the noisy warnings.
Ratchet is such a good word for it.
Ratchet is a good name/pattern. It is also grandfathering.
It is similar to how code coverage can be done. Old coverage may be low e.g. 40%, but may require 80% coverage on new lines, and over time coverage goes up.
I wonder if there has ever been a sneaky situation where someone wanted to use forbiddenFunction() really bad, so they remove the call elsewhere and tidy that up, so they could start using it.
Interesting, props for coming up with a good name.
But it's weird to me to call this a "ratchet", and not just a custom lint rule. Since it sounds exactly like a lint rule.
The hard-coded count also sounds a bit like something that I would find annoying to maintain in the long run and it might be hard to get a feeling for whether or not the needle is moving in the right direction. - esp. when the count goes down and up in a few different places so the number stays the same.. you end up in a situtation where you're not entirely sure if the count goes up or down.
A different approach to that is to have your ratchet/lint-script that detects these "bad functions" write the file location and/or count to a "ratchets" file and keep that file in version control.
In CI if the rachet has changes, you can't merge because the tree is dirty, and you'd have to run it yourself and commit it locally, and the codeowner of the rachet file would have to approve.
at least that would be a slightly nicer approach that maintaining some hard-coded opaque count.
We did something similar with TypeScript strict mode Turned it on per file with a ratchet count, and over a few months, the whole codebase was strict without ever blocking anyone
I built something like this that we use both for migrations and disallowing new instances of bad patterns for my mid sized tech company and maintain it. Ours is basically a configuration layer, a metrics script which primarily uses ripgrep to search for matches of configured regexes, a linter that uses the same configuration and shows any configured lint messages on the matches, a CI job that asserts that the matches found are only in the allowlisted files for each metric, and a website that displays the latest data, shows graphs of the metrics over time, and integrates with our ownership system to show reports for each team & the breakdown across teams. The website also has the ability to send emails and slack messages to teams involved in each migration, and when the configuration for a migration includes a prompt, can start a job for an agent to attempt to fix the problem and create a pr.
I like the idea of ratchets, but the implementation needs to be good for them to work nicely.
> If it counts too few, it also raises an error, this time congratulating you and prompting you to lower the expected number.
This is a pain and I hate that part. It's one of the things that isn't even a big deal, but it's regularly annoying. It makes leaving things in simpler than removing them - the good act gets punished.
One way to make this better is to compare the count against the last merge base with the main branch. No need to commit anymore. Alternatively you can cache the counts for each commit externally, but that requires infra.
Shameless self-promotion, but my own post on Ratchets from a few years back: https://thraxil.org/users/anders/posts/2022/11/26/Ratchet/ Similar basic idea, slightly different take.
I think this could be handled by an open rewrite rule [0], with the side effect that it could also fix it for you.
When the calls to THE FORBIDDEN METHOD are eventually replaced and the method removed, we can bury the ratchet.
It's like looking for "linter regressions" rather than test regressions.
Love it! …but of course I’d worry about a diff that added one offense while removing another, leaving the net sum the same. Perhaps the author handles this? You want to alert on the former and praise on the latter, not have them cancel out through a simple sum. Admittedly it’s a rare sounding edge case.
The more trad technique for this would be to mark the offending line with # noqa or # ignore: foo. Another way is to have a .fooignore file but those are usually for paths or path globs to ignore.
I like the author’s idea[1] of having the “ignore” mechanism next to the linter codebase itself, rather than mixed in with the production codebase. Adding the files and line numbers for known-offenders to that code could be a useful alternative to a simple sum?
Perhaps more robustly, some kind of XPath like AST syntax to indicate which parts of the codebase have the known problem? It feels just as fragile and could quickly get over complicated.
At the end of the day an online comment has always done it for me. With Python, Meta’s libcst is an excellent and fast way to get an AST that includes comments. It’s the most robust tool I’ve found but you can just use built-in ast.py and ad-hoc file:line parsing too.
https://github.com/Instagram/LibCST
[1] Sorry to be a fanboi but Antimemetics is amazing!
[dead]
I built a ratchet system for ESLint originally that we’ve extended it to work with TypeScript, Terraform, and Biome linters.
integrating with each linger is complex but it pays dividends - it’s so handy to be able to write a new lint rule or introduce an off-the-shelf rule without needing to fix all existing violations.
We maintain allowed error counts on a file-by-file basis which makes it easier for developers to understand where they added the new violation.
blog post: https://www.notion.com/blog/how-we-evolved-our-code-notions-...