alt Hacker NewsNice, I'm really interested in using this for simple semantic search in a native desktop application.
Any comparisons with other tiny embedding models? Did you start from MiniLM-L6 because it's an especially good model in its class? It's hard to figure this out since all you provide is "Retrieval (SciFact NDCG@10)".
But the claimed performance seems way off, I get only 35 emb/sec in firefox on a i5-4570 rather than 400/sec. Is there an issue with falling back to a non-SIMD path? I'll try a native Rust binary next.
Same! I’m trying to find small models that can embed effectively to enable BM25/hybrid search over a large number of documents for a personal information repository. Ideally, it should run on consumer hardware.
bge-small-en-v1.5 is one that is comparable and what we’re working with for now.