alt Hacker NewsCan't you just look at memory bandwidth for generation?
Replies
stingraycharles • yesterday at 10:38 PM
It’s more nuanced than that, but yes that’s the biggest factor.
Can't you just look at memory bandwidth for generation?
It’s more nuanced than that, but yes that’s the biggest factor.
You also need to look at memory size (to store the model). And processing can still be the bottleneck.