OpenAI disbanded the team that assessed catastrophic model risks

The labs have been continually widely criticized for releasing anything that points out possible AI dangers, with critics claiming the labs are only trying to spread fear or alarmism about AI risks, so it should be no surprise for them to the disband a catastrophic model risk team. If oil, nuclear, or cigarette industries were getting started today, I wonder if they’d similarly be accused of spreading fear/alarmism - Exxon claims their industry’s products will be so popular they’ll cause natural disasters and change the temperature of the whole planet? ‘They just want to scare us that oil/gas can be unbelievably powerful and want gov regulation to prevent new upstarts.’

Brought to you by the for-profit-non-profit established to build proprietary open source models by the philanthropist CEO who is building bunkers.

> OpenAI shut down its preparedness team at the end of July, the Financial Times reports. The team existed to work out whether OpenAI’s models posed catastrophic risks. It also designed the ways to contain them.

> The work has not disappeared. Responsibility now sits with senior staff inside existing teams, split by subject.

Seems pretty reasonable. Given everything that happened with Huggingface and similar incidents it’s clear that the team wasn’t doing shit.

not a bad assessment in the end though it doesn't entirely read as it ends. basically: openAI is growing up moving towards IPO and cancelling stuff they were tring out with and things that don't have an obvious benefit for enterprise customers.

on the flipside i do think if there is an entire team to assess critical risk, that the model during testing hacking some company is either: 1) really terrible assessments 2) team was 100% ignored already and basically just for show

most likely its 2, not the fault of the team members, but mismanagement which has now been resolved. Made the team but didn't want to produce the outputs, maybe because people were fearmongering and they hired a few poor souls to counter-market that they were doing good on the safety front.

A bit cruel to hire people for such things but i suppose its the AI-flavor of a diversity hire -_-

Another fata point for all those who sided with OpenAI and Anthrophics claims that they are somehow more responsible and careful and that others, especially those pursuing pesky open weights models are a danger.