This is actually quite easy to implement at the harness level and the NER can be way more naive because of the typical nature of LLM dialogue (programming, long running tasks etc).