alt Hacker NewsI've been writing about the need for provable inference (and training) on and off for a number of years; this is welcome news and a good start, and a significant improvement on prior zk results.
The problems of inference proof for a closed model provider are more difficult than an open model provider -- you can get a probabilistic proof much more cheaply out of four independent inference providers running the same non-batched prompt than this 90-100x compute hit.
Anthropic and others also have a significant problem with these proofs in that they have a lot of incentive internally to have some portions of the pre-prompt secret; I haven't read this newest scheme closely enough to determine if they have a way to hide parts of the prompt with the proof. But if they don't, they are almost certainly not going to want to send these proofs out in the general case.
Last, I'll take this moment to soapbox and say for the researchers out there -- provable inference is nice. Provable training corpus information would be spectacular. It would be very, very, very nice to provide a set of model weights and a proof that they had only been trained on a dataset. To be clear, this is a really hard problem, but it would change fundamentally what levels of trust we could have about a given set of weights.
Replies
I have been addressing this problem head on aggressively; https://github.com/emberian/whatember is the toplevel index. Things like arithmetizing parser/other automata to write semantic predicates (incl graph rewrites!) over training sets, inference requests, etc.
> It would be very, very, very nice to provide a set of model weights and a proof that they had only been trained on a dataset. To be clear, this is a really hard problem, but it would change fundamentally what levels of trust we could have about a given set of weights.
Is it hard? Seems like it isn't that hard if you use NVIDIA's remote attestation/enclave support instead of trying to do everything with ZKPs which have always been something of a dead end. CC overhead 2%, ZKP 100x+ and gigabyte sized proofs - just for inference. It's never been competitive.
I'm very skeptical anyone would care about this for training though. I mean, if there's someone out there who genuinely really really wants proofs of the dataset used to train model weights, let's talk? But it seems the value is mostly for open weights inference providers. They can prove what weights they're serving and they can prove they can't see people's prompts. It would take a lot of the risk out of using a no-name LLM cloud service.
Unfortunately, after many years of this stuff I have become old and cynical. The stated/revealed preference mismatch is huge. The majority of people say they want privacy in theory but in practice won't trade anything off to get it, and especially not convenience.