> It's not absurd to think that a company would use its own harness or an open source harness and/or put a proxy in to route requests variably. For two months, they have engineers using different models unbeknownst to them and they measure things like how often the engineers have to correct or redirect the model. At the end of two months, they might find little difference in performance and switch to a much lower cost model.

That's a great product idea!