alt Hacker NewsPeople will put up with a lot. People are celebrating that you can run models on a CPU at single-digit tokens per second. You think there won't be a single person that can put up with that and also be dangerous/etc?
People will put up with a lot. People are celebrating that you can run models on a CPU at single-digit tokens per second. You think there won't be a single person that can put up with that and also be dangerous/etc?
It's highly impractical. Imagine someone breaking into a house to steal something or otherwise, and they can only take 1 step every 20 seconds. They won't be getting anywhere, when even a child in the house can notice them and go call for help at 1 step/2 seconds and said help will come at 5 steps/second.