logoalt Hacker News

jftugatoday at 3:19 PM0 repliesview on HN

I wonder how this compares to my M4 air with 10 GPU cores and 32 MB of RAM. My system can only run ~14B sized models at any reasonable speed. The accuracy of these sized models can be underwhelming. I am looking forward to a time when it would be nice to run models locally at a reasonable price, at a reasonable speed and with reasonable accuracy. I don't think we are there just yet.