logoalt Hacker News

observationistlast Thursday at 9:30 PM2 repliesview on HN

The idea that tokenization is what they're for is absurd - you're talking a tenth of a thousandth of a millionth of a percent of efficiency gain in real world usage, if that, and only if someone bothers to implement it in software that actually gets used.

NPUs are racing stripes, nothing more. No killer features or utility, they probably just had stock and a good deal they could market and tap into the AI wave with.


Replies

adastra22last Friday at 1:45 AM

NPUs aren't meant for LLMs. There are a lot more neural net tech out there than LLMs.

show 1 reply
microtonallast Thursday at 9:39 PM

I think they were talking about prefill, which is typically compute-bound.