alt Hacker NewsWhich specific model? and are tools available? I would say that there is lots of RL with tool use, but that the reasoning traces are not always perfect. In general, models think about things that might be wrong, but will recover from this poor line of thinking.
This was using llama-serve on my local which by default gave it current time & unhelpfully "get_info" which is specific to the browser
The full request was
``` {"messages":[{"content":"What animal lays the biggest egg?","role":"user"}],"return_progress":true,"sse_ping_interval":1,"stream":true,"tools":[{"function":{"description":"Returns the current local date and time in ISO 8601 format, with the IANA time zone name","name":"get_datetime","parameters":{"properties":{},"required":[],"type":"object"}},"type":"function"},{"function":{"description":"Get runtime info (OS name), may call when user asks about local files or shell commands","name":"get_info","parameters":{"properties":{},"required":[],"type":"object"}},"type":"function"}],"reasoning_format":"auto","reasoning_control":true,"backend_sampling":false,"timings_per_token":true} ```
Disabling tools removes the bit about mentioning instructions.
Just an interesting detail I noticed.
Do y'all have any penalty/regularization w.r.t. reasoning traces or is it pure SFT -> RL?