alt Hacker NewsI was doing something similar where I saved user input/model output in a multi-depth node style storage system (each depth having more precise details) with the focus on the model having accurate user fed information. I was mostly focused on retrieval of accurate / useful information based on user query (injecting the database node as additional, high confidence information)
Once this(Zero-mem) passes it's peer review, I may have to see if my system can handle something similar instead/in addition.
I'm quite excited to see growth in these different ways of eliminating token's.
Long winded aside, @langs, have you published your work on this?
https://github.com/AttemorySystem/attemory/ stars and issues are welcome :)
Using attention for retrieval was inspired by a comment I saw in here long time ago: Prediction and retrieval are two sides of the same coin; to predict better, you must retrieve more accurately.
I'm still working on the improvement of algorithms, my tests shows the performance and accuracy will be improved a lot in the next release.