We could save petabytes of cache storage with Zstandard and Pingora

Curious how this application scheme compares to filesystem + transport compression. You'd end up potentially compressing and decompressing more often but the higher software doesn't need to know what's happening and the compression happens in kernel space.

ie btrfs

You could also layer on out of band dedupe and probably push out cache updates with btrfs snapshots although maybe that ends too convoluted

Why not serving files compressed if the client supports it even though the origin served an uncompressed file?

> We initially considered limiting transcoding to popular content

Weird, I would have compressed cold content instead, if the goal was to save on CPU time during decode.

I'm confused by how this affects range requests. Without compression, those can be easily satisfied by reading the relevant part of the cached complete file. But how are they handled now? The article claims "range requests remain unchanged", but I don't see how that's possible if the cache no longer stores the uncompressed data.