logoalt Hacker News

DragonBoostertoday at 12:20 PM1 replyview on HN

These models have open weights, but at the moment most flagship models are practically accessible only through third-party model providers. The main exception is models in the ~30B parameter range, which can still be run on consumer-grade GPUs. That said, even consumer GPUs have become increasingly expensive and difficult to justify in recent years.


Replies

mirekrusintoday at 12:37 PM

You can definitely go above 30B on consumer hardware – 2x gpus, spark, mac, half byte quants etc.