alt Hacker NewsHey, very inspiring article, Redis engineer here. How do you work with static allocation on variable query structure, and row counts that can explode depending on the data shape?
And isn't there a benefit for small allocations on advanced memory allocations that you can't leverage if all is working in big page allocations? Do you implement memory allocations from scratch or leveraging existing allocator on top of these memory blocks strategy somehow?
Replies
I've long used similar static allocation models in analytical database kernels. These are even more susceptible to widely varying demands on memory. There are few practical limitations or caveats to this allocation model and it has strong advantages for both robustness and performance engineering. Memory organized as pages is compatible with small allocations, and is more or less how classic allocators work.
The runtime allocation is type-aware, workload-aware, and schedule-aware. The last is most important. There are two places that can act as a sink for heavy memory demands: storage (i.e. paging to disk) and network (e.g. streaming results). These have their own limitations because I/O bandwidth is finite. Effectively, your allocation rate is equivalent to available I/O bandwidth.
The most powerful lever you have to manage this is total control of the schedule. Demands on memory are created by a set of operations or queries visible to the software. The scheduler doesn't incrementally execute these operations randomly, it continuously selects execution based on the availability of memory or bandwidth to absorb the allocation demand of the operation. The scheduler has the ability to control the allocation rate to instantaneously match availability.
This is essentially the very old idea of "optical buffering" -- treating fiber optic cables as RAM -- taken to its logical architectural conclusion.
The caveat is that this requires direct I/O in userspace, which places limits on software architecture. But if you care about performance, you'd be using this type of software architecture regardless.
Thanks! We use streaming data structures.
For example, if you take a look at our LSM compaction, regardless of the table size, we compact at the 512 KiB block granularity, and everything is streaming.
The same principle applies everywhere.
In our experience writing TigerStyle (and for all our internal code and tooling, not only TB as DBMS), we’ve never had a scenario where static allocation was not applicable or didn’t produce a better design.
You also tend to become more memory efficient, not less. Again, since you’re streaming. (You’re not allocating a massive buffer, just because a file is multi-GiB.)