alt Hacker NewsWhy is it like that, currently? There is no information added by a vector index compared to the original text. And the text is highly redundant and compressible with even lossless functions. Furthermore a vector index is already lossy and approximate. So conceptually it is at least possible to have an index that would be a fraction of the size of what is indexed?
There is some information added, depending on the vector db and context (some systems will add permissions related metadata so that the LLM won’t pull chunks that the user didn’t have access to).
The vector itself is pretty large (512 dimensions).
The chunks have an overlap (iirc 30% but someone feel free to correct me).
I don’t _think_ the data is typically compressed (not sure why but I assume performance).