logoalt Hacker News

zamadatix • 01/21/2025 • 0 replies • view on HN

That's the general task but the hard part is having the pile of local machines with nearly a TB of VRAM to distribute it on. You'd need over 30 3090s worth of GPUs to run those models.