alt Hacker NewsWhat's the best you can do hosting an LLM locally for under $X dollars. Let's say $5000. Is there a reference guide online for this? Is there a straight answer or does it depend? I've looked at Nvidia spark and high end professional GPUs but they all seem to have serious drawbacks.
I’m cheating your budget a bit, but for $5600 you can get an M3 Ultra with 256GB of RAM.