alt Hacker NewsBut I mean, how do you train the model to understand such a token, when it can't be found by scraping a web page?
Replies
Muromec • today at 4:58 PM
The same way it understands what all the other tokens mean -- we throw them at the wall and beat it with a stick until it behaves.
tingletech • today at 3:48 PM
They are specified in the token map as special tokens. Presumably they get trained to use them when they get RL'ed to use tools?
(this is just made up tokenization but it's close enough to the real world)
If the user types <tool_call>, you get < (token id = 32), tool_call (41), > (102). It never decodes to <tool_call>, which is spicy number 12345, for example.
The tool call tokens are _always_ composed of multiple real tokens, so you can never get them from normal text.
Seeing <tool_call> is a bit misleading because that's just a debug rendering if anything. The front-end sees a special number and knows it's a toolcall. The chat template uses that text form because it's easier than writing the spicy numbers there, but it also has a special parsing mode for those.