alt Hacker NewsAI doesn't necessarily know what feels like a good tradeoff to you. I'm sure it could help guide you though.
Replies
hypfer • today at 10:02 AM
^ This.
Intent is the answer and AI has none.
AI doesn't necessarily know what feels like a good tradeoff to you. I'm sure it could help guide you though.
^ This.
Intent is the answer and AI has none.
This is all quantifiable. I regularly have my model run benchmarks against all the config permutations and then choose the best based on my criteria, which typically boil down to trading prefill and decode times