alt Hacker NewsWouldn't it be possible to just fix a single activation, just set a continuous input to what the harness knows the role actually is. Models could then be trained to trust that input and not other signals about roles.
Replies
jackb4040 • today at 4:07 PM
The models wouldn't need to adapt their training at all; this is a secondary model trained on their activations. It's electrodes stuck into the model's brain. The original article I posted is built on top of a regular LLM (OpenAI's GPT-OSS)
I suspect there are many excellent solutions along these lines available to the labs training the models.
I wonder how well one could do on a conventional model with careful input formatting, e.g. JSONL where every line has bounded length and is something like:
It could need a bit of fine tuning to get this to work well.