alt Hacker News> MLA/CSA/HCA
Aren’t these techniques all “lossy” compression, and one of the reasons people complain about loss in quality as the context size grows larger?
> MLA/CSA/HCA
Aren’t these techniques all “lossy” compression, and one of the reasons people complain about loss in quality as the context size grows larger?
Indeed they are all lossy. Not sure how much they contribute to the quality loss in long context though. I got a 700k session with DSV4 Pro (official API), and the model was still coherent and didn't make any tool call error.