Cool demo, but even the smallest model does 2.5 tps on my 7900xtx which is not great.

lnenad • yesterday at 8:37 AM • 3 replies • view on HN

k3liutZu • yesterday at 8:50 AM

I get 9-10 tps on my MacBook Pro. Though the small model seem really limited

➕ show 1 reply

LorenDB • yesterday at 3:50 PM

18.8 tps on an 11th gen mobile i7 here.

christkv • yesterday at 8:58 AM

Hmm I get 44,7 tokens/s on the smallest 1.7b one in the browser on my m1 max.

alt Hacker News