Open sourcing Dicer: Databricks's auto-sharder

Seems weird to call it sharding since it's not sharding indexed datasets or anything like that. Is this just a tool to mitigate Databricks’ internal service-scaling challenges?

Does anyone else have something similar?

What are some use cases that you found are useful?

[dead]