I tried some 1-bit, 2-bit, and bonsai quants against closed eval sets. They were essentially useless for my case. The little errors accumulate and send the whole output off track quickly.

If you had some use case with very small output sequences they could be interesting to try. I think dropping down to a 9B-class model would produce better results for most cases.