I built a simple emacs package based on this idea [0]. It works surprisingly well, but I dont know how far it scales. It's likely not as frugal from a token usage perspective.

0: https://github.com/dmitrym0/dm-gptel-simple-org-memory