alt Hacker NewsYou don’t want to do that for anything you want to be able to vary, but they do something similar with a “soul document” for things they always want to apply.
Replies
energy123 • today at 2:00 PM
If it's a fine tuning step at the end, why is the need for it to vary a problem? Can't you run the fine tuning, test for regression, and deploy the weights in a day?
I think the more likely reason is it doesn't work as well as in context learning. Otherwise they would prefer to avoid polluting context and degrading performance.
➕ show 1 reply
In this token-mania frenzy that has taken hold of the industry, I guess solutions like "soul document" and "system prompts" will continue for a while, and once the industry matures a bit we'll go back to things like LoRA[1] and control vectors[2][3].
The other explanation may be that these AI labs may be expecting more government scrutiny, and "here's a document" would probably go better than "here's some vector representation of our values" when talking to politicians.
[1] https://arxiv.org/abs/2106.09685
[2] https://vgel.me/posts/representation-engineering/
[3] https://transformer-circuits.pub/2024/scaling-monosemanticit...