alt
Hacker News
MrScruff
•
yesterday at 9:46 PM
•
0 replies
•
view on HN
I get around 20 tok/s, 4 bit quant, MTP, 4 bit KV cache quantisation. On an M4 Pro 48Gb.