alt Hacker Newsgood luck running a 2.4T model on any local hardware. it’s not gonna happen. the arrow is to specialized hardware at least for the smartest models
Replies
sir, I'm not running a multi billion dollar code base; I just want my nose wiped and a clean fork of whatever repo might be the target of supply chain attacks, and a few nicissities.
I don't need 2.4T to do that; I'm doing it with 35B or 27B. If they get me a model in ~80B with a A5B or A7B, that will be the end point.
It's bizarre people, by themselves, believe all these parameters are getting them much more.
Lets be serious: if we as a civilization really wanted the advancements promised, we'd find the 1000 best scientists and give them free access to these models while the rest of us get personal GPUs for specific use cases.
But instead, we have to endeour this penis measuring contest for the infinite bikeshedding of the universe.
Yes but someone who has access to that kind of hardware can distill down to a smaller model that is specialised for a specific task. I don't need my local model to be an oracle for everything, I want a coding AI, one that knows medicine, another that recognises objects in my security camera, etc.