alt Hacker NewsLinear scans break at scale. Partitioning by prefix and using pooled 1024-entry blocks is the right move to prevent 2 GB worst-case index bloat.
Linear scans break at scale. Partitioning by prefix and using pooled 1024-entry blocks is the right move to prevent 2 GB worst-case index bloat.
Yes, they do, but the extent to which that matters varies greatly. Accuracy/time tradeoffs exist (and are moderately common at $WORK right now). If you provably can't do better than a linear scan (and benefit from the increased accuracy from doing so at a business level), you might as well lean into it and choose a dead-simple, CPU-friendly algorithm for your problem.