alt Hacker NewsI created a pruned experts model of the q2 quant, while it gave good performance on limited hardware there was severe quality degradation.
I created a pruned experts model of the q2 quant, while it gave good performance on limited hardware there was severe quality degradation.