logoalt Hacker News

espenbtoday at 11:47 AM1 replyview on HN

Yes, indeed. The list operation is expensive. The S3 spec says that the list output needs to be sorted.

1. All filenames are read. 2. All filenames are sorted. 3. Pagination applied.

It doesn't scale obviously, but works ok-ish for a smaller data set. It is difficult to do this efficiently without introducing complexity. My applications don't use listing, so I prioritised simplicity over performance for the list operation.


Replies

luke5441today at 11:53 AM

Maybe mention it somewhere as a limitation, so it is not used for use-cases where listing is important and there are many objects?

Listing was IMO a problem with minio as well, but maybe it is not that important because it seems to have succeeded anyway.