logoalt Hacker News

cqql • 01/21/2025 • 1 reply • view on HN

What kind of resources do I need to run these models? Even if I run it on a CPU, how do I know what amount of RAM is needed to run a model? I've tried reading about it but I can't find a conclusive answer, other than downloading models and trying them out.


Replies

sumedh • 01/21/2025

On a Mac with 16 GB ram you can rum the 8B models.