logoalt Hacker News

kristianpyesterday at 11:13 PM2 repliesview on HN

I tried "select * from items limit 10" and it is slowly iterating through the shards without returning. I got up to 60 shards before I stopped. Selecting just one shard makes that query return instantly. As mentioned elsewhere I think duckdb can work faster by only reading the part of a parquet file it needs over http.

I was getting an error that the users and user_domains tables aren't available, but you just need to change the shard filter to the user stats shard.


Replies

piperswetoday at 2:23 AM

Doesn't `LIMIT` just limit the amount of rows returned, rather than the amount read & processed?

show 3 replies
ncrucesyesterday at 11:40 PM

That's odd. If it was a VFS, that's not what I'd expect would happen. Maybe it's not a VFS?

show 1 reply