Issue is that both harness and specific model matters a lot in what type of instruction works best, if you were to use Anthrophic's models together with the best way to do prompting with Codex and GPT models, you'd get a lot worse results compared to if you use GPT models with Codex, prompted in the way GPTs react best to them.

I don't think people realize exactly how important the specific prompts are, with the same prompt you'd get wildly different results for different models, and when you're iterating on a prompt (say for some processing), you'd do different changes depending on what model is being used.

Agree, for now im using dotagents by sentry to handle a lot of this.

And why would they ever let switch?