alt Hacker NewsThis works especially well if your embedding model was trained to perform well with quantized embeddings. Binary + hamming distance = incredibly fast.
This post is from 2024 but I wrote about using this technique in https://emschwartz.me/binary-vector-embeddings-are-so-cool/
Hamming w/xor+popcount is the only thing I can make numpy do faster than float32 dot products :)
int8s, float16s are all fairly slow. I suppose it’s because BLAS does float32/64 very fast.