Memorilla: Latent Semantic Memory for LLMs

So if I'm reading this right, it's basically a specialized embedding of a RAG, and then a model in the middle takes the agent's queries and returns responses in a form that is injecting into the main model's input token stream directly i.e. skipping actual text. And the injection format is INCREDIBLY token efficient. Sort of injecting a vibe rather than quotable text. Their results of "good on concepts, less good on quotes/exact text retrieval" seem to intuitively line up with that.

I am finding that semantic ~= lexical when you have something like a frontier model driving. Using full text search engines is still the most compelling approach for handling large document sets.

I really struggle with the semantic / embedding concerns when we have a generative model that can reverse engineer a Mario 64 rom dump to run on my toaster oven. These models know how to compose Lucene & other FTS query strings in such a way that we are effectively getting semantic search over purely lexical systems.

[flagged]

[flagged]