alt Hacker NewsWell 8%-10% saving without measured quality degradation is not nothing, specially now that newer models seem to use more tokens than previous ones.
Also it was tested on reasoning low, i'd have liked to have them tested on higher reasoning levels.
Well 8%-10% saving without measured quality degradation is not nothing, specially now that newer models seem to use more tokens than previous ones.
Also it was tested on reasoning low, i'd have liked to have them tested on higher reasoning levels.
Ask, what is the function of grammar in language? I would argue two major functions. Forcing an encoder to marshal ideas into a coherent structure. Preserving the integrity of that structure when communicated and decoded.
Grammar sounds off? Was the message correctly understood; is there enough left to apply error correction and regenerate the intended message?