P99 0 ms* autocomplete for 240M domain names

This autocomplete suggests domains that don't exist. You can just type garbage and it will suggest something, but then if you go there, there are no records.

It seems like one purpose of an autocomplete box is help you avoid typos, so that makes it less useful.

Using keyup makes no sense and is inconsistent with user expectations. For triggering actions (which includes normal typing), you only ever use keydown. (Well, there’s one exception for reasons unclear to me: activating a button by pressing Space. That triggers on keyup like how clicks are on release, while Enter triggers on keydown.) Keyup is limited to things where you’re constantly reacting to the state of a key, as is common in games.

This affects the functionality, too. It is in fact introducing latency by using keyup instead of keydown. Feels bad.

Unfortunately this approach doesn't feel that great down here in Australia, definitely a function of latency.

I think you could get a lot closer by framing this as an optimization problem, where you use the full alphabet dictionary, but add a residual prediction which aims to cover as much of the remaining domain name tree as possible weighted by popularity. This tree could then be pre-baked and stored with the same system. This would probably get you p99 0ms even in Australia.

Why not just trigger the fetch on keyDown and show it as soon as the response arrives, as usual?

The time it takes to press a key is a reasonable target to aim at for API latency I suppose, but it is still an arbitrary target. Waiting to display until keyUp just adds more latency if your API is faster. Having it synced with keyUp doesn't make it feel more immediate to me.

If you’d like to reduce the network latency further you can store each trie node as a file, naming it conveniently the prefix path to that node. Then dump the few hundred million files onto R2.

Now the traversal can be done completely via CDN lookups!

Tangentially, I built something similar a few years back, at link-archive.org: https://web.archive.org/web/20220127233707/https://link-arch...

3B existing URLs extracted from CommonCrawl, with instant search results. It was a fun project but didn't serve much real-world purpose besides curiosity and discovery. So I eventually ditched it, primarily because the link DB was a whopping 500 GiB in size, too much to just keep hosting.

I just used SQLite FTS5 as the backend search engine. Just a few lines of code, but immediate response from a 0.5 TiB DB. SQLite is amazing.

>I've designed the API to first search Tranco (the head), and then CZDS (the tail) if necessary.

How do you know if it's necessary? If Tranco returns less than eight?

I just typed a random sequence of the characters, long enough to be certain such domain doesnt exist. No only, the browser send an autocomplete request for every keystroke but for each request it returned a set of proposed domain names (which I'm 100% certain doesnt exist). At this point, how do I understand which results are legit and which are fake? Also, it would be nice to highlight the typed part in the result set so I can visually see what matches exactly.

Clever but that’s not how we measure latency.

Looks more like 500ms?

How are the icons (for the domains) fetched so quickly?

KeyDown events don’t work great for mobile, though.

Autocomplete aside, this is a pretty nifty tool.

Pretty sure we mean < 1ms rather than actually instantaneous.

its pretty clever but what happns when someone pastes a domain or uses IME or voice input? the api being that fast is still impressive.

[flagged]

[dead]

[flagged]