logoalt Hacker News

rockorageryesterday at 3:46 PM6 repliesview on HN

Author of the project here! I have a little write up on this here: https://rockorager.dev/log/lsr-ls-but-with-io-uring


Replies

ChuckMcMyesterday at 7:08 PM

Nice writeup. I suspect you're measuring the cost of abstraction. Specifically, routines that can handle lots of things (like locale based strings and utf8 character) have more things to do before they can produce results. This was something I ran into head on at Sun when we did the I18N[1] project.

In my experience there was a direct correlation between the number of different environments where a program would "just work" and its speed. The original UNIX ls(1) which had maximum sized filenames, no pesky characters allowed, all representable by 7-bit ASCII characters, and only the 12 bits of meta data that God intended[2] was really quite fast. You add things like a VFS which is mapping the source file system into the parameters of the "expected" file system that adds delay. You're mapping different character sets? adds delay. Colors for the display? Adds delay. Small costs that add up.

1: The first time I saw a long word like 'internationalization' reduced to first and last letter and the count of letters in between :-).

2: Those being Read, Write, and eXecute for user, group, and other, setuid, setgid, and 'sticky' :-)

tavianatoryesterday at 4:27 PM

My bfs project also uses io_uring: https://github.com/tavianator/bfs/blob/main/src/ioq.c

I'm curious how lsr compares to bfs -ls for example. bfs only uses io_uring when multiple threads are enabled, but maybe it's worth using it even for bfs -j1

show 1 reply
cb321yesterday at 8:16 PM

At those time scales, you would be better off using `tim` ( https://github.com/c-blake/bu/blob/main/doc/tim.md ) than hyperfine { and not just because that is your name! Lol. That is just a happy coincidence by clipping one letter off of the word "time". :-) } even though being in Nim might make it more of a challenge.

jeffbeeyesterday at 4:01 PM

How much of the speedup over GNU ls is due to lacking localization features? Your results table is pretty much consistent with my local observations: in a dir with 13k files, `ls -al` needs 33ms. But 25% of that time is spent by libc in `strcoll`. Under `LC_ALL=C` it takes just 27ms, which is getting closer to the time of your program.

show 1 reply
benreesmanyesterday at 6:05 PM

This is fantastic stuff. I'm doing a C++ project right now that I'm doing with an eye to eventual migration in whole or in part to Zig. My little `libevring` thing is pretty young and I'd be very open to replacing it with `ourio`.

What's your feeling on having C/C++ bindings in the project as a Zig migration path for such projects?

show 1 reply
dangyesterday at 5:51 PM

(Thanks - we'll make that the main link (since it has more background info) and include the repo thread at the top as well.)