A short post, but it's every easy to read. Great job Daniel. Looking forward to more posts about this.
One recent case we had is that a developer noticed the memory wasn't being returned to the OS, and sent a pull request that forces `malloc_trim`. There are some Python projects that also do this call periodically, but while it works for `malloc` allocated objects, it doesn't work jemalloc, mimalloc (default for thread-free builds, if I'm not mistaken). It doesn't break either, but still...
The `malloc_trim` does seem to help producing a better memory usage pattern, but it's accompanying a `gc.collect`, so I'm being cautious and asking for more tests/benchmarks/etc.
Another developer has found a dictionary that seems to be the actual source of our problem, so we may not have to add `malloc_trim`, but maybe that'd be something interesting for a future post if you haven't already dug into that direction :)
alt Hacker News
A short post, but it's every easy to read. Great job Daniel. Looking forward to more posts about this.
One recent case we had is that a developer noticed the memory wasn't being returned to the OS, and sent a pull request that forces `malloc_trim`. There are some Python projects that also do this call periodically, but while it works for `malloc` allocated objects, it doesn't work jemalloc, mimalloc (default for thread-free builds, if I'm not mistaken). It doesn't break either, but still...
The `malloc_trim` does seem to help producing a better memory usage pattern, but it's accompanying a `gc.collect`, so I'm being cautious and asking for more tests/benchmarks/etc.
Another developer has found a dictionary that seems to be the actual source of our problem, so we may not have to add `malloc_trim`, but maybe that'd be something interesting for a future post if you haven't already dug into that direction :)