logoalt Hacker News

themanualstatestoday at 5:02 PM1 replyview on HN

That’s useless without describing WHY you chose those flags, and how you did the optimisation…


Replies

halJordantoday at 6:24 PM

The switches are all in the -h of llama.cpp (although the maintainers have a tendency to use the word in its definition). The actual values are essentially just what alibaba recommends. So you just need their model card. I would not call it highly optimized, more appropriately tuned.

show 1 reply