logoalt Hacker News

throwdbaawaytoday at 7:20 AM1 replyview on HN

Different sessions. With https://github.com/fairydreaming/llama.cpp/tree/dsv4, 1M context with DSV4 Flash takes less than 6GB of VRAM. I can't run DSV4 Pro, but it should take less than 9GB of VRAM for 1M context, based on the numbers shared in https://arxiv.org/html/2606.19348v1.


Replies

scope2093today at 8:06 AM

Thank you for the links/docs. I'm quite excited to try it myself.