alt Hacker NewsUse of closed models is unprofessional, and depending on your field negligent. The fact that it has been widely normalized does not make it less so.
You're handing over your (presumably your customer/employers) data to an unaccountable third party which has demonstrated itself willing to commit criminal acts, and to take other people's data without permission. Your ability to continue to perform this work can be withdrawn at any time for any (or no) reason. You have little ability to validate that the work is being performed as expected and isn't being silently nerfed or outright subverted based on competitive considerations, bribes, overactive 'safety', or cost management.
Outsourcing to a black box would be a reasonable expectation if you asked a non-professional to perform the work. A professional should be able to account for the tools they use.
Replies
I think that's a bit strong of an assertion. How many people use copilot daily under an enterprise agreement? I don't necessarily disagree in spirit, especially given the questionable data sanitization around the recent Navier-Stokes announcement, but most companies disclose huge amounts of data regularly to hopefully-trusted third parties. I think the internet -- and a good share of the world's commerce -- would grind to a halt if we suddenly stopped. Setting up, securing, and maintaining local models for even a small user base is non-trivial and there is way more demand than supply for that skillset right now.
Anyway here's how Cloudflare orchestrates AI reviews at scale: https://blog.cloudflare.com/ai-code-review/
AI token machine go brrr!
> Your ability to continue to perform this work can be withdrawn at any time for any (or no) reason.
Thanks to the fact that there are no widespread stories about this actually occurring in practice, at least not yet, people do not take it as a relevant risk at the moment.
> You have little ability to validate that the work is being performed as expected and isn't being silently nerfed or outright subverted based on competitive considerations, bribes, overactive 'safety', or cost management.
Yes, I agree that this is a real concern that many people might rightfully have. And I am unaware of any way to mitigate this concern while using black box AI models. Because the only thing that their creators can do is to tell their customers: "trust us". But there is no way to objectively verify whether they serve tainted AI model responses or not.