My dream binary format is schema driven, as compact and efficient as Capt Proto or such, but just optionally embeds the entire schema into the message. Then we can write a vim plugin that just opens the file in human readable form without having to fish for the schema. Whenever I am using binary formats, it's because I have a list of millions of objects of the same types. Seems to me that you may as well tack 1KB of schema onto a 2GB message and make it self-describing so everyone's life is easier.
As another user suggested, Avro is something to look into.
You could have a look at Avro (https://avro.apache.org/) and Yardl (https://microsoft.github.io/yardl/).