alt Hacker NewsWait so this makes it so I can use my DDR5 as well as my VRAM combined? This is actually sick if so. Maybe I will actually have to go out and buy some more DDR5 (currently only have 32GB...)
Wait so this makes it so I can use my DDR5 as well as my VRAM combined? This is actually sick if so. Maybe I will actually have to go out and buy some more DDR5 (currently only have 32GB...)
Yep, thats it what it does. Only works with nvidia.
The difference it does use safetensors, and not gguf's. But it does dynamically requant to int4 8 or bf16.