alt Hacker NewsNumber of params isn’t really the relevant metric imo. Top models don’t support local inference. More relevant is tokens per dollar or per second.
Replies
lm28469 • today at 11:24 AM
It does since you can run this model locally on a < $3k machine
Number of params isn’t really the relevant metric imo. Top models don’t support local inference. More relevant is tokens per dollar or per second.
It does since you can run this model locally on a < $3k machine
Its an open source model, why wouldn't it be relevant for people who want to self host.....