alt Hacker NewsCan I ask what did you buy? I don't have that money now but I wouldn't knpw what to buy anyway.
Replies
jliptzin • yesterday at 9:21 PM
Blackwell 96gb + 128gb system ram. Not powerful enough for GLM 5.2 but I may upgrade.
➕ show 1 reply
Can I ask what did you buy? I don't have that money now but I wouldn't knpw what to buy anyway.
Blackwell 96gb + 128gb system ram. Not powerful enough for GLM 5.2 but I may upgrade.
I can run most models 100B and under on my MacBook Pro with Ultra 3 and 128GB of RAM at 25-70tok/sec-ish, and it was around $5k.
I don’t think I can run GLM 5.2 since it requires around 256GB of memory and the inference is probably too slow, but the future (planned) Apple M7 may be able to. The leaks say it will support up to ~700GB of RAM.
The models under 100B are kind of dumb as a brick and aren’t that useful unless you’re really bad at coding imo. They can’t really be trusted not to hallucinate so they’re not even good for data processing.