logoalt Hacker News

alex7oyesterday at 6:01 PM1 replyview on HN

Turboquant on 4bit helps a lot as well for keeping context in vram, but int4 is definitely not lossless. But it all depends for some people this is sufficient


Replies

carlovalentiyesterday at 10:13 PM

[dead]