How could they implement it? Try testing a bunch of models (closed and open sourced) and then seeing which one gives the best returns for it's cost? And then how do they check if it's being properly used, I have read of people just throwing their token budgets to the fire so that they show high usage for KPIs, while the most obvious cases of "X do this very wasteful thing" will be culled quickly (hopefully), I don't see how non-technical management can see through the thinnest layer of malicious compliance
How could they implement it? Try testing a bunch of models (closed and open sourced) and then seeing which one gives the best returns for it's cost? And then how do they check if it's being properly used, I have read of people just throwing their token budgets to the fire so that they show high usage for KPIs, while the most obvious cases of "X do this very wasteful thing" will be culled quickly (hopefully), I don't see how non-technical management can see through the thinnest layer of malicious compliance