logoalt Hacker News

po1nttoday at 7:45 AM2 repliesview on HN

I love how claude focuses on exfiltrating the data "I need cha for charlotte". This could be solvable with some kind of low powered safety agent that would check claude's reasoning for anything immoral/unsafe. We could call it common sense. It won't fix the problem completely but at a certain point it would be easier to trick human than a machine.


Replies

khalictoday at 8:32 AM

A paper came out lately showing that exposing a classifier to the chain of thought actually hurts the final verdict

sinfulprogenytoday at 8:11 AM

"the security hole in the agent could be solved with another agent"

I think the point the article is making points in another direction.