logoalt Hacker News

skrebbelyesterday at 8:04 PM3 repliesview on HN

What the hell is a "safety score for violence"?


Replies

I-M-Syesterday at 8:47 PM

It's making sure AI condemns violence perpetuated by people without power and sanctifies violence of those who have it.

show 2 replies
murat124yesterday at 8:11 PM

I asked an AI. I thought they would know.

What the hell is a "safety score for violence"?

A “safety score for violence” is usually a risk rating used by platforms, AI systems, or moderation tools to estimate how likely a piece of content is to involve or promote violence. It’s not a universal standard—different companies use their own versions—but the idea is similar everywhere.

What it measures

A safety score typically evaluates whether text, images, or videos contain things like:

Threats of violence (“I’m going to hurt someone.”) Instructions for harming people Glorifying violent acts Descriptions of physical harm or abuse Planning or encouraging attacks

show 1 reply