alt Hacker NewsAt first I thought this was work on the far more important problem of proving that LLM output is provable and correct given some source of truth. But it's about the trivial problem of 'proving' which model is being used.
Who will find this useful? I can't really think of a use for it, because if you don't trust the provider of your models, it's game over really - you're already giving them all sorts of private data, you're relying on their service, you have to trust them to some extent and this doesn't really help because unless you are running the service yourself. Seems like more of an internal problem for Anthropic than something that would interest others.
Replies
For example, there are Chinese resellers that promise to provide access to Claude (which isn't officially available in China) and defraud users by substituting something else. They're obviously sketchy in all sorts of ways, but in theory, this would prevent one kind of fraud.
It might also provide an additional guarantee for users of Open Router and Fireworks.
>Who will find this useful?
It's worth reading past the opening paragraph before asking these kinds of questions:
>When a user sends a request to an AI model, today they have no way of knowing for certain that the model they selected is the one that generated the response. For everyday uses, in which users mostly care about the output and not necessarily which specific model produced it, this is a less pressing concern. But there are several cases where such assurance would be useful. For example, independent evaluators who test AI models before and after release need to know that the model they are evaluating is the model being served. Clients who use a model through a third-party routing service want assurance that the tokens came from the model they paid for. And if a cyberattacker modifies the token stream while pretending to be a given model, a user has no way to detect it from the output alone.
This is from the first paragraph in the Introduction.
Customers care about model output being correct, Anthropic cares about model output being generated by Anthropic model.
This solves for the latter with a bunch of handwaving for the former. The end goal is gonna be FUD for cheaper model routing because that obviously is at odds with their biz model.
Off the top of my head, it is useful when there are content-level proxies between the client and anthropic's API server, of which there are many these days.
The main benefits are:
1. it proves your models are running correctly. FP ops tend to deviate in small amounts between hardware providers and tensor ops even more so due to the relatively new and unstable standardisation environment atm. So it proves your HW configuration is delivering outputs correctly. i.e. it's a good test for what is currently a very non-trivial validation environment.
2. it proves your external provider is running the inference they said they are. i.e. it eliminates the concerns of model providers running quantized versions silently behind the scenes since quantization breaks the proof and you need to use a new, updated proof signature for the quantized version.
3. likewise for fine tuned or tampered models. Or outright substitution for another model (ex the accusation by OAI and Anthropic that Kimi and DS and co are periodically forwarding requests to their platforms to harvest responses for knowledge distillation).
All in all it's a nice chain of trust utility if it can evolve out of the toy stage and reach production viability (since rn it's limited to int-only weights and 1k context which is practically useless atm).