alt Hacker NewsIt feels like file formats and protocols is the wrong prism through which to view this. What users actually want is the ability to move as much data as possible from point A to point B, whilst remapping it as much as possible to meet the different features and identities of point B. A unified data format is actually a hindrance for meeting that need because it boils it all down to a lowest common denominator. A simple data pump utility that just copies data back and forth, with lots of special case logic like "X has a character limit of 5000, Y has a character limit of 500 so I'll truncate and add a link to the original post on X", is going to work far better for this use case.
More generally I feel like you kinda missed the point of filing systems in your first writeup. The moment you say "posts don't have natural names", ok, that's a serious problem that should put a halt to the entire concept. A file is nothing but a natural name for a byte array and a little bit of metadata. That's why we think of file managers and explorers when we think of what files are all about: lists of names. It's also why people directly use files for relatively few things - they were a big deal in the early days of personal computing because the killer apps for it were direct translation of physical office tasks. It's mostly but not entirely possible to come up with natural names for office documents (the gap is in suffixes like "draft final v3__real final draft.docx"). That's why we use analogies for 1950s paperful office objects like files and manila folders to begin with. But most computing tasks even back then weren't oriented around documents, they were oriented around database records and the user interface was just screens, or possibly direct SQL. It's a remarkable gap in our infrastructure that there is no agreed on file format for a SQL result set. Maybe Arrow is that format.
>A unified data format is actually a hindrance for meeting that need because it boils it all down to a lowest common denominator.
Exactly, which is what I'm saying in the article:
We could try to put every app developer in the same room until they all agree on a perfect lexicon for a post. That would be an interesting use of everyone’s time.
(That's meant to be slightly sarcastic.) Then the article says:
For some use cases, like cross-site syndication, a standard-ish jointly governed lexicon makes sense. For other cases, you really want the app to be in charge. It’s actually good that different products can disagree about what a post is! Different products, different vibes. We’d want to support that, not to fight it.
The point isn't having a lowest common denominator format. It's to enable precisely the granularity of formats that app developers want. The default one is "each app has its own formats". But what changes is that
1. Other apps can still read/write in other apps' formats. This makes the landscape competitive: if some app is going down a bad road, it's easy to fork their product with all the data already in there.
2. It is possible for multiple app developers to use the same format where it makes sense. As linked from the article, https://standard.site/ is a good example of that.
>More generally I feel like you kinda missed the point of filing systems in your first writeup.
I'll admit that I used the metaphor to highlight a specific thing, and names in particular don't play a big role in that thing (what plays a big role is links). The reason I picked filesystem as a metaphor is because it highlights how file formats mediate interactions between apps. This is exactly how lexicons mediate interactions between social apps. That felt like the backbone of the argument.
It's true this isn't describing a filesystem exactly. Maybe a web of identity-addressed hyperlinked JSON is a more precise way to say it. But then nobody knows what that means. I'm happy with my choice of metaphor.