So they are sharding by time/day?
I have a similar project right now where I am scraping a dataset that is only ever offering the current state. I am trying to preserve the history of this dataset and was thinking of using the same strategy. If anyone has experience or pointers in how to best add time as a dimension to an existing generic dataset, I'd love to read about it.