logoalt Hacker News

Havoc • today at 12:43 AM • 0 replies • view on HN

The bonsai quants are not that awesome. Yeah they're fast and they retain the "27B" but in practice the perform similarly

5GB of bonsai weights end up being similar to 5GB of QAT weights even when one is derived from a 27B and the other from an 8B