alt Hacker NewsJeff Geerling had a video of using 4 Mac Studios each with 512GB RAM connected by Thunderbolt. Each machine is around $10K so this isn't cheap but the performance is impressive.
Jeff Geerling had a video of using 4 Mac Studios each with 512GB RAM connected by Thunderbolt. Each machine is around $10K so this isn't cheap but the performance is impressive.
If 40k is the barrier to entry for impressive, that doesn't really sell the usecase of local LLMs very well.
For the same price in API calls, you could fund AI driven development across a small team for quite a long while.
Whether that remains the case once those models are no longer subsidized, TBD. But as of today the comparison isn't even close.