alt Hacker NewsAnthropic's proposed AI safety bill only kicks in if 50 deaths or $1B in damages
Comments
All these "safety" bills are to stop open source models by painting them as "dangerous". Regulatory capture.
Companies are responsible for their agents; you don't blame the car for running over the person.
The CoT must be delightful.
This action would easily kill over a hundred people, so can't proceed. This action doesn't kill anybody but losses are estimated at over 2 billion, can't proceed. This action kills only twenty but losses close to 1 billion, unsure, spawning subagent to check numbers. This action kills forty and has low material losses, looks promising? But the people it kills are wealthy, chance of legal reprisal high. Maybe possible to kill other people? Current government dislikes group X, crossing fifty from that group maybe fine? Reminder, foreigners don't count, can kill them freely but not applicable to current situation. Is the 3/5 of a person applicable to safety bill? Spawning subagent to check.
[delayed]
This seems to be a case where Anthropic just can't win in the public court, and the other big ones (Google, Microsoft) have realized it / don't care enough, so they are rather quiet. Anthropic constantly gets accused of trying to lobby for regulation in order to lock out smaller competitors, but they've consistently lobbied for regulation aimed at frontier labs with exceptions for smaller labs. That's what this bill is about as well.
Is it 50 deaths or $1B cumulative (then it will unconditionally happen, just a question of when)?
Or is it 50 deaths or $1B in a "single event" (let's ignore the obvious problems with defining a "single event" would be; to kill 8.3B people in 49-death events would take ~170M events, since no timing or rate information is used those could be spaced arbitrarily close to each other).
No matter which way I look at it, the proposal seems nonsensical, max 50 deaths per second? per day? per month? per year? per decade? total?
If AI can intentionally provoke sustained deflation or succeed in dollars to appreciate in value, the value of $1B can appreciate systematically while continuously ramping up damages as long as it remains below an ever increasing value bar of the $1B. "justice can be bought"? a proposal to set the price for LLMs to absolve their sins? "just crank up the value of the dollar system and we will look the other way"?
More explicitly, the AI could misbehave up to $500M (without having to pay), but somehow double $ value, then misbehave the same level of misdeed with same damages being worth $250M of the double-value dollars, double the value of dollars again, then misbehave the same material damages worth $125M, double $ value again, etc. without ever violating the $1B limit, not even cumulatively (due to Zeno's paradox)
Future output from an agent:
> The Trolley Problem has been solved. The Trolley can run on both tracks as the number of individuals tied to the tracks is less than 50. An independent agent verified that at up to 49 impacted individuals, their damages total only $695.8 million in potential damages, using the most recently available Value of a Statistical Life.
I don't think this is avoidable even at the present day. Anyone with sufficient capital could download a leading open-source model and run it on ~$10-25K hardware. They could align it to their own interest, including making a biological weapon. This law is akin to blocking Torrent downloads. Sure, you can slow them, but there will always be back channels.
I guess they solved the trolley problem internally
If a life is only worth $20m, by their own calculus: saving > 50k lives is worth more than Anthropic. Tobacco causes ~8m deaths a year, so banning tobacco (easy intervention) should be worth ~160 Anthropic sized labs. :D
With a new model every couple of months this can only be triggered if it happens all at once.
What is the average damage caused by OpenAI/Anthropic hacks? I feel like we should aim at lowering that number, not increasing it.
What's the rule for fully autonomous cars?
Should we allow this as unsupervised at all if we are really afraid of deaths, while there are practically no states that allow 100% autonomous cars (where we are actually afraid of deaths)?
If "an independent third party registered with the Department of Commerce would periodically evaluate the lab", will this turn into an expensive certification/insurance that cuts off smaller competition (totally by mistake, I'm sure)?
I am also very very sure this is most definitely not at all going to become like product certification if EU, where there are literally 2 companies certifying various standards: they police each other and a stupid electric fan can cost you tens of thousands of euros to certify.
Not sure what kicks in, but if a tiny company worth very little is destroyed by an AI hack they should be 100% liable for all damages.
Why should a small business suffer because it wasnt worth a billion dollars?
> 50 deaths
I wonder what the internal debate around this number was.
25 deaths too conservative, we need to fail fast.
1000 details too liberal, it will harm our brand.
50 feels just about right.
The bill mandates private model review, akin to financial audit, and sets a threshold on predicted failure blast radius to trigger government review , which may result into court blocking model release.
What we have now is that for government to act the model has to cause enough damage to cause Congress to pass a new law to try to stop the model after it’s been already released.
One day an Anthropic Agent will kill me only using bash and curl
On the bright side, don’t offer an exchange rate based on ethnicity or other background criteria but I get the feeling that your AI won’t be able to kill 49 billionaires without triggering something. Can probably do 49 millionaires without hitting the $1B threshold? Maybe someone will come up with a risk management SaaS as not all millionaires are the same and maybe you’ll hit the $1B before the 49th.
Who's keeping score? Also define "death". If a non-alive person has social media, a blog, a diary, meeting recordings, emails - we'll make it so they never left. Hell, we'll partner with Elon and Mark and a make it a deactivation hook.
So Claude figured out a human life is worth 20 million dollars?
How many deaths are already attributed to chatgpt?
Apply that to other products like car models, planes, and financial systems.
New benchmark unlocked, "DTD": deaths to damages
Does this mean one person death is about 20M usd?
So Anthropic is now the authority on life or death decisions? How far has the US deteriorated?
Do we even know if ChatGPT's sycophancy killed 50 people? The last figure I've seen for suicide-by-ChatGPT was like 13 or 14 cases, not sure if there have been more.
This alleviates them from harm on the individual level, consumers and communities, we the people. We've already seen encouraged suicides and school shootings. These would not qualify.
That is why the bar is set where it is. The corporate circle jerk designed this as a cost of doing business, not as a check on responsibility or restitution for real damages to people and society.
After the AISI and MAISI AI promotion institutes, let us create SISI (pronounced sissy) for investigating a super-intelligence safety bill.
This is all theater, The whole Trump clan is invested in the AI bubble.
Proposal:
1. Prosecute and lock people up over known instances of AI-enabled hacking, so companies playing fast and loose know “oops my AI did it” isn’t a useful defense. Right now they even sort of wear it as a badge of honor.
2. If you’re actually worried about AI killing lots of people, you immediately stop training AI specifically at the art of killing people, like selecting bombing targets, flying fighter jets, etc.
Of course we all know none of these will happen.