logoalt Hacker News

tommy_axleyesterday at 3:34 PM1 replyview on HN

Pick a decent quant (4-6KM) then use llama-fit-params and try it yourself to see if it's giving you what you need.


Replies

gunalxyesterday at 6:27 PM

I habe found llama-fit sometimes just selects a way to conservative load with VRAM to spare.