logoalt Hacker News

andai • today at 4:39 PM • 0 replies • view on HN

>Through our experiments, we found that you should consider switching harnesses when switching models to maximize your experience.

I've seen this for years, and seen several papers that studied it: the same prompt will help with one LLM and harm performance for a different LLM on the same task.

And on benchmarks which test LLM x Harness, they find that LLMs tend to do way better with their "own" harness. (Or in the case of Chinese LLMs, the harness of the LLM whose training data they borrowed :)