alt Hacker NewsLooks very much like a format that should just have been gzipped JSON.
Don't use binary formats when it isn't absolutely needed.
Replies
>Looks very much like a format that should just have been gzipped JSON.
For application file formats that require storing binary blob data such as images, bitmaps, etc , many in the industry have settled on "SQLite db as a file format": (https://www.sqlite.org/appfileformat.html)
Examples include Mozilla Firefox using sqlite db for favicons, Apple iOS using sqlite to store camera photos, Kodi media player uses sqlite for binary metadata, Microsoft Visual C++ IDE stores source code browsing data in sqlite, etc.
Sqlite db would usually be a better choice rather than binary blobs encoded as Base64 and being stuffed into json.gzip files. One of the areas where the less efficient gzipped JSON might be better than Sqlite is web-server-to-web-browser data transfers because the client's Javascript engine has builtin gzip decompress and JSON manipulation functions.
Every format is binary in the end, you are just swapping out the separators.
I personally subscribe to the Unix philosophy. There are really two options, binary or plain text (readable binary due to a agreed standard formatting). All other formats are a variation of the two.
Additional a binary format suits makes sense when the format is to be used for mobile devices which may not have much storage or bandwidth.
Strong disagree. I like binary formats because I can just fopen(), fseek() and fread() stuff. I don't need json parser dependency, and don't need to deal with compression. Binary formats are simple, fast and I need a way smaller buffer to read/write them normally. I don't like wasting resources.
> Don't use binary formats when it isn't absolutely needed.
JSON in particular isn't very good [0] but I'd also argue that text formats in general aren't great. Isn't it a lot better that an int32_t always takes exactly 4 bytes instead of anywhere between one and eleven bytes (-2147483648)?
JSON is for humans to read and maintain, not machines to read/write - and machines read/write millions of times more than humans do.
Using JSON / HTTP for backend communication (i.e. microservices) is madness when you consider how much overhead there is and how little value having it be human-readable delivers.
I choose binary formats over JSON almost every time I can. JSON sucks big time.
Uh. You want to store assets in JSON? Why? You generally want asset packs to be seekable so that you can extract just one asset, and why would you want to add the overhead of parsing potentially gigabytes of JSON and then, per asset, decoding potentially tens of megabytes of base64?
... and that is why all 'modern' software is incredibly memory and CPU intensive...
[flagged]
It's not new. BOMStore is a format inherited from NeXTStep. JSON didn't exist back then.
Also, it's a format designed to hold binary data. JSON can't do that without hacks like base64 encoding.
Binary file stores like this are very common in highly optimized software, which operating systems tend to be, especially if you go looking at the older parts. Windows has a similar format embedded in EXE/DLL files. Same concept: a kind of pseudo-filesystem used to hold app icons and other resources.