logoalt Hacker News

Tepix • yesterday at 2:59 PM • 1 reply • view on HN

Sounds good. I saw that you use the FP8 version of the model. Do you also quantize the KV cache?

Replies

sacrelege • yesterday at 5:02 PM

no I don't, since there seem to be a silent degradation bug