logoalt Hacker News

chzblcktoday at 5:39 AM1 replyview on HN

you running a server or local llms with a need for 512?


Replies

arjietoday at 5:41 AM

Server stuff. Nothing interesting. Supermicro H11 + Epyc 7xxx + RAM. I have a 6x4090 setup for local LLMs and I got myself a 128 GB M4 Max laptop thinking I'd do that, but if I'm being honest I need to get rid of that hardware. It's sitting idle because the SOTA ones are so much better for what I want.