alt Hacker NewsWell I wouldn't call it a low bar, since some of the edits were quite complex. And 1M context in less than 6GB of VRAM is truly impressive, but somehow this gets way less attention than the crappy turbo quant from Google.
Well I wouldn't call it a low bar, since some of the edits were quite complex. And 1M context in less than 6GB of VRAM is truly impressive, but somehow this gets way less attention than the crappy turbo quant from Google.
I'd like to understand this please. Why would the 1M context be kept in VRAM if you're using DSV4 Pro through the API? Or did you refer to different sessions?