The models are already massively over trained. Perhaps you could do something like initialise the 2 new token sets based on the shared data, then use existing chat logs to train it to understand the difference between input and output content? That's only a single extra phase.

You should be able to first train it on generic text once, then duplicate the input layer and fine-tune on conversation.