alt Hacker NewsIn case anyone else clicked in wondering what counts as "real time" for this:
> On a Pi 5 (16GB), Q3_K_S-2.70bpw [KQ-2] hits 8.03 TPS at 2.70 BPW and maintains 94.18% of BF16 quality.
And they talk about other hardware and details. But that's the expanded version of the headline claim.
Replies
make3 • last Wednesday at 6:39 PM
I wonder what "94.18% of quality" means
Someone should make a version of the Hacker News homepage that is just LLM extracts of key article details like this.