logoalt Hacker News

kennywinkertoday at 5:27 AM0 repliesview on HN

180b * 1.76 bits per weight = 39.6 gigabytes.

Best you could realistically run in 32gb is like 28gb, or a 127B param model