alt Hacker NewsAvoid Mini-Frameworks
Comments
Having been around in the industry for a while I'm seeing abstractions being misused all the time. Being guilty of it myself also when I was younger.
For me, the only purpose of an abstraction is to reduce complexity, but often times I'm seeing it being used to reduce repetitiveness, which often times replaces well understood more verbose code with less understood less verbose and less flexible alternative. For me, as a team lead, easy to read code is far more important than subjectively perceived elegant abstraction that everyone then has to learn how to use, and potentially fight with.
In many cases I have noticed people jumping into abstracting away a complexity right away, often times ending up with a leaky or inflexible abstraction. To those people I say, do that painful thing at least 10 times, then think about abstracting it away, since then you probably have some level of understanding about the pain you're trying to alleviate and all the nuances that comes with that domain.
This was great. I enjoyed the alternative definition of "framework" as something that introduces new concepts (differing from the more common idea that it's code that calls YOUR code.)
This article reminded me of two classic pieces of writing.
The first is 20+ years old now: Joel Spolsky's law of leaky abstractions:
https://www.joelonsoftware.com/2002/11/11/the-law-of-leaky-a...
One of the reasons these mini-frameworks lead to problems is that they leak. They don't cover every case which means you inevitably have to fully understand what they are doing for you in order to debug them or work around their limitations. Your cognitive load goes up!
The second is Will Larson's "Migrations: the sole scalable fix to tech debt."
https://lethain.com/migrations/
The OP complains that they've never seen a technology migration at Google that finished. Will advocates for migrations to be driven by a team that both coaches other teams on the migration and then, crucially, finish the job themselves to make absolutely sure it gets done to 100% completion.
Yes, this little-known blog post from 2015 with a similar line of thought was one of the most influential of my career: https://tomasp.net/blog/2015/library-frameworks/
The modern variant of this is "platform vs service". At my previous company, it seemed like the only path to promotion was "platform" something something. So every org was in the middle of some way over-budget rewrite of their services as some big bloated platform thing, incompatible with every other team's platform, with less functionality than their old services offered. I don't know why leadership sees all these failures and seems to think "ooh, that's exactly what I want for my team too!" But you can't get a promotion if you're not platformizing.
In my career, I've found that this problem crops up the most when a team is unable to make impactful changes to a system that they depend on. It's so much easier (and requires less collaboration and fewer approvals!) to build an abstraction over some core system than to actually fix the core system, even if fixing the core system is always the better choice.
I was very guilty of this as a young go-getter engineer! Why try to convince another team that something should be fixed if I can just paper over it?
I never heard the term mini framework before but I like it. Applied within that context it makes sense. I was at Google 2011-2013 through an acquisition and witnessed some of what the author is describing. Actually something very specific comes to mind. There was a SQL library built on top of BigTable, Google's internal columnar data store, I think it was called Megastore [1]. The team implementing a new product decided to use it over BigTable directly because what it would mean for ease of development but it turned out as soon as they needed to do any data migration it was locking the entire database (or I guess the table? Aka BigTable). Anyway at the time this was a major issue and they had to revert to BigTable best practices as opposed to what this library was doing. Because essentially it was a library layer, not some new data store. Eventually Google built a different SQL based data store to replace the whole thing which you can see the open source version known as CockroachDB, which some ex-googlers invented.
Moral of the story, abstractions always fail the edge cases, especially at scale. But when the entire eng org adopts something, it works much better. Everyone has to be bought in. Which at Google scale is hard.
https://research.google/pubs/megastore-providing-scalable-hi...
The problem in big tech is that: Sending upstream patches to THAT team (that owns the original framework) is like pulling teeth.
Those people have their own OKRs and promo desires.
Often times, sending patches to OSS orgs are easier.
So naturally, extension to the original framework happened.
Basically, premature abstraction / abstraction at the wrong boundaries coupled with the political power to force adoption of the bad abstraction. Yep. That’s an annoying attribute of working at a big corp. I prefer working in tiny teams and companies for this and many other reasons.
I'm not sure I can accept the findings of this article. It seems to me that all the concerns and issues can be summed up to bad design. A mini framework should be designed to be easy to adapt and extend, and any layer on top should be just as easy to further adapt and extend. That to me is the crux of this, "avoid frameworks that are difficult to adapt and extend".
Making good abstractions is hard.
And it is very easy to start feel like you know you have a great abstraction when you don't. And unfortunately it's easy and fun to make abstractions, kinda like making babies. And has kind of similar weight to it too.
I just have to say.. stay safe out there.
This is a curse of lots of software companies:
** mini-frameworks is a realization of the creator's mental model, but it's not everyone's mental model**
People being smart enough to make their own understanding work well - but not smart enough to see they are just pushing their way of doing things and not working on something “generally understood”.
It’s a simple math problem. And it is also Conway’s law that says all software design follows the organization that built it—that is all software design is political.
A framework calls you. You call a library.
A framework constrains the program. A library expands the program.
It’s easier to write a library that is future proofed because it just needs to satisfy its contract.
It’s harder to write a framework because it imposes a contract on everything that depends on it.
Just like it is hard to write tort law without a lot of jurisprudence to build out experience and test cases, it is hard to write a framework from only one use case.
No one likes lawyers because they block you from doing what you want. This is the problem with frameworks.
However the government likes laws because they block you from doing what you want. Same with whomever is directing engineering that wants all other programmers to work in a consistent way.
> Start fresh. Don't build a wrapper around the existing framework, build your own from scratch.
I keep thinking "Just use GraphQL" because it's basically GQL at this point...except jankier.
As a rule of thumb, "magic" is a code smell. Libraries should be preferred over frameworks whenever possible.
A toolbelt of small utility-like composables are often easier to maintain and reason about. This results in added explicitness (i.e. less magic, fewer surprises).
Personal experience shows that the immediate efficiency gains of a framework often get diminished in the face of all the hacks people introduce later, just to work around the remaining 10% of cases that the framework did not anticipate or traded-off against.
Please note this is a comment based on personal experience and professional preference.
BOCTAOE.
Frameworks promised faster development cycle, lest code and uniform codebase. But over the years, this proved to fail to deliver except in certain minority of cases.
I think modern programmers nowadays understand that it is much better to take the path of libraries than frameworks as it provides them with the same functionality of a framework(which is just a bundle of libraries), but with the freedom of implementing their code however they want, unlike wit ha framework, which forces certain structure and style as frameworks had to made certain decisions for the programmer in order to be a functional and comprehensive tool. This lack of freedom will usually bite most programmes LATER, when it is too late to go back and refactor code or change style and whatnot.
And that inherently also makes small frameworks even less usable than the larger ones.
YMMV, but not really.
> We often hear that "people leave managers, not jobs." But sometimes, people leave jobs despite loving their managers.
These two aren’t really mutually exclusive. Your manager may be extremely friendly and accommodating to you personally (as it seems like the author’s manager was), but part of a manager’s job is growing and supporting their reports with their career goals. If you’ve spent years majorly underleveled like it seems this person did, your manager is failing you. No matter how much of a nice person they might be, they’re not doing their job well if you’re attempting to grow at the company and aren’t succeeding.
So yes, I do think that this person still left a manager. He left a manager who wasn’t meeting the needs he had to stay at the company.
I wrote this article to reflect a pattern I observed (and hate) while working in Google, but I'm sure this is not a company specific problem. Would be interested to hear other people's stories :)
I can be found here:
Mini-frameworks would be dangerous at a place like Google where smart people think they are smart. I think they're heaven sent for smart people who think they're stupid.
Distributed (multithreaded, concurrent, ...) systems are a counterexample that are highly vulnerable to snake oil. In normal software it makes sense to build up from a small set of intellectually coherent primitives. In those cases you inevitably end up with poor performance and usually reliability if you try that. Java started out with a snake oil approach to threading (synchronized!) and Doug Lea talked some sense into them and now we have java.util.concurrent which has a rich set of primitives which are pragmatic and meet the real requirements, not a vision of purity.
On the other hand, If it was a mini-framework to pound out numerous not-so-simple HTML form applications it could greatly enrich your life and your team's.
I don't really understand the concept. What is the definition of "mini-framework" ? The author should have given a few examples.
I have the impression that he confuses "obscure" with "mini". Either framework or library..
This reminds me a bit of using LLM frameworks like langchain, Haystack, etc., especially if you’re only using them for the chat completions or responses APIs and not doing anything fancy.
This is one of the rare HN thinkpieces that's actually correct. You cannot remove essential complexity. You can shuffle it around, but it never goes away; you will have to deal with it. You can try to hand it off to someone else, like another team. But that actually makes it worse, because there's still some critical piece you depend on that's hidden behind the silo walls.
Rather than trying to hide things to "ease adoption", the correct answer is to educate people. Devs hate learning things. But once they learn the new thing, the pain goes away, and the results are better. The more you try to avoid it, the more problems you create later.
This is why I use Ruby on Rails. Shared constraints and boring conventions age better than clever mini-frameworks built around one team's mental model.
> I work in Google Ads infrastructure in the past four years
I am glad they were up front about this. Saved me a read.
Somehow, somewhere there is a pleasant balance between DRY and non-DRY which is different for everybody. God forbid having a colleague who sees a thing repeating and slaps an abstraction over it at whatever cost because DRY!
This. This. This. I currently work in a codebase where so much code has been abstracted away for “cleanliness” that it’s impossible to understand what code is actually running.
The worst is when three lines of completely standard code (immediately understandable to anybody inline) get „helpfully” lifted out into a utility function.
It's why I've always eschewed stuff like flask even when I felt like Django was going to be overkill. The problem is that, especially for web apps, everybody needs the same stuff. You need to handle cookies and forms and auth. You need to be able to inspect and manage your data. You need to be able to do tokens and password resets.
At the end of the day. You end up cobbling together a bespoke, worse version of Django anyway.
The right number of frameworks in a company is log10(number of employees). Go over this limit, and everyone needs to learn too many frameworks with too little support, examples, tools, and documentation.
"shared infrastructure owned by dedicated teams" must be nice, I have to say
> Introduce new concepts that doesn't exist in the original stack
That is also true for "macro" frameworks.
> Wraps around the company/org-shared tech stack or framework
That is often also true for "macro" frameworks.
> Creators claim that the framework "magically" solves many problems, and push more people to use it
That is often also true for "macro" frameworks.
---
It is not clear from the reader's perspective what actually characterizes a "micro" framework. It's also not clear why the size is the issue here, when all complaints seems to be about design or quality.
Is googletest a micro or macro framework? Is google/zx a micro or a macro framework? Give us some clarifying examples. Actual things people can look for, not internal unknowable projects. There must be some exceptions too (silver bullet rules don't exist), mention them.
Also, rethink the title. Maybe "makeshift frameworks" is better terminology, as it more accurately reflects the problem that is described in the content.
[dead]
> real and only difference between a library and a framework, is whether it introduces new concepts
This isn't what is normally understood in software engineering by those terms.
A library is something you call.
A framework is some kind of application scaffolding that normally calls you.
You can use more than one library. You normally only have one framework in-process.
I found the blog post a little hard to parse. Is it an argument against wrapping frameworks, or wrapping libraries?
I agree that wrapping frameworks is fraught with danger. I can't quite agree for wrapping libraries. Wrapping libraries makes a lot of sense if you're only using a tiny fraction of the library functionality, the breadth of the wrapper's API is much smaller than the library's API, wrapping it enables you to substitute it (whether for a smaller / faster / whatever dependency in the future, or for testing, etc.), and so on.