> It would be very, very, very nice to provide a set of model weights and a proof that they had only been trained on a dataset. To be clear, this is a really hard problem, but it would change fundamentally what levels of trust we could have about a given set of weights.

Is it hard? Seems like it isn't that hard if you use NVIDIA's remote attestation/enclave support instead of trying to do everything with ZKPs which have always been something of a dead end. CC overhead 2%, ZKP 100x+ and gigabyte sized proofs - just for inference. It's never been competitive.

I'm very skeptical anyone would care about this for training though. I mean, if there's someone out there who genuinely really really wants proofs of the dataset used to train model weights, let's talk? But it seems the value is mostly for open weights inference providers. They can prove what weights they're serving and they can prove they can't see people's prompts. It would take a lot of the risk out of using a no-name LLM cloud service.

Unfortunately, after many years of this stuff I have become old and cynical. The stated/revealed preference mismatch is huge. The majority of people say they want privacy in theory but in practice won't trade anything off to get it, and especially not convenience.