alt Hacker NewsTechnically speaking when you send the prefix “The capital of France is “ into an LLM it will also produce probabilities across its whole vocabulary.
Technically speaking when you send the prefix “The capital of France is “ into an LLM it will also produce probabilities across its whole vocabulary.
… which they could provide in their APIs but are vehemently opposed to because it makes distillation much easier, and faster.