alt Hacker News> * Do not provide assistance to users who are clearly trying to engage in criminal activity.
I don't know what we want to call this, but in my opinion, having to convince your tools is not computer science.
Kind of amusing that we made it as far as we did as a species not really being able to explain how the human brain does it's most amazing tricks and then we just replicated it while still not really understanding the emergent capabilities all that well.
Replies
These system prompts are not the only safety layer that these models use. There's other more deterministic filters in place both on input and (streaming) output.
In one way you’re right, of course, but if you look at Fable, for example, that uses similar guardrails, it’s downright impossible to discuss these things.
It is my understanding that having a secondary model whose sole purpose is to trigger based on guardrails is the way this is usually done.
> I don't know what we want to call this, but in my opinion, having to convince your tools is not computer science.
Bit of a mouthful, but how about just calling it "auto-regressive language modelling".
Feeding it stuff to auto-regress on is obviously your main control vector.
Apparently RL-trained models like rewards too. PHB's can use "you've gotta work all weekend, but you'll get comp time when it's fixed".
Prompts are not good “guardrails” anyway.
Mmm, quite.
> I don't know what we want to call this, but in my opinion, having to convince your tools is not computer science.
My vote is "machine psychology".
We didn’t replicate the human brain. We built systems that can statistically approximate some of what the human brain might output in certain limited situations.
That prompt is there for legal reasons. Non deterministic output is the expected outcome.
> in my opinion, having to convince your tools is not computer science.
If you think the system is a tool and not an intelligent, conscious entity (I think you are correct in this), then you cannot reasonably think of input to that system as an attempt at persuasion, even if that input happens to consist of English prose. Treat it as a nondeterministic programming language, and the objection evaporates.
> not really being able to explain how the human brain does it's most amazing tricks and then we just replicated it while still not really understanding the emergent capabilities
I think you could say much the same about, say, a pacemaker. "Replicated" is overstating the case quite a bit.
It's a hack but doing things the 'proper' way is at least 1000x harder so whatever.
NLP guys were right all along :)
The alternative is Claude-style "safeguards" aka censorship, which:
1. doesn't eliminate the possibility of a jailbreak anyway
2. frequently has false positives, triggering on innocuous requests, which is just really annoying
Not saying that we can't (or shouldn't) do better than Grok, but I really don't know what the best solution is here...