alt Hacker NewsYou are completely mad if you are running any kind of valuable workload on Runpod community or Vast. Hosts are absolutely peeking inside the VM's (you can see it openly discussed in their Discords).
Replies
I hold the inverse risk assessment- I think acting on a peek risk is much higher with larger operations such as OpenAi or eg Amazon -as they've allegedly done the same on their product marketplace and deciding on generic products, in addition to open source-alike service offerings.
Larger entities have the resources, the inclination and the protective legal teams to act much differently than a resource tight n-tier platform operator
> (you can see it openly discussed in their Discords).
Can you explain? I haven’t used Runpod and don’t have time to search through a Discord to figure out what someone is talking about.
Could you please share screenshots?
What are places in your view to rent GPUs that are trustworthy they will not peek in your opinion?
I only use their serverless stuff, but you can choose which security and compliance you want for their serverless stuff.
A lot of Vast.ai GPUs are also firmware-limited, so it's a big gamble when you run your workloads. You also can't get a cluster.
It's fine for comfyui workloads or simple LLM inference, but you lose both privacy and stability.
Sam is welcome to peek at my generative furry porn all he wants ( ͡° ͜ʖ ͡°)
The 1.8x difference between the cost in AWS and Azure is still relevant though.
Honestly I think every one is peaking at everything, I don't want to be slanderous but I think even the more trusted cloud providers peak, though they do so more quietly and carefully.
Could confidential computing help here?
There's a lot of use cases that are not "commercially sensitive" that would require data center level privacy. E.g. a good percentage of academic research and experimentation, long running "boilerplate processing" for monitoring logs for e.g. a bird watching system or large scale web crawling where the individual website itself is mostly public. There are also use cases where the data itself is useless without other data to correlate. E.g. many log analytics use cases, if you are missing the main database data itself then the logs have zero value when intercepted. There are also use cases where you are doing general industrial level hotdog/not hotdog classification where unit economics is the most important and there's no real value to privacy e.g. on a factory floor (though sabotage is a bigger risk here).
You are ignoring the real usecases here. Not everything is large scale, secretly, b2b enterprise applications. Instead, many consumer applications are fully fine with being out in the open.
As an example, midjourney use to just publish ever generated image to their website.
If you are just doing inference things are public/open from the start, you dont have to worry much about the stuff you are talking about here.
[flagged]
[dead]
There's got to be a ton of valuable workloads that don't care much about secrecy.
Who cares if someones spying on the process of fine tuning a model to help us pick strawberries? Even if they could somehow find a competitor to sell it to - the competitor wouldn't be interested because the risk of violating trade secret law is surely greater than whatever they can glean from our training run...