I have a 'node/container' abstraction at the infra/engine layer which is essentially either a cloud VM or a local podman container. The engine/infra layer can spin up more of these as needed. I have a relatively beefy dedicated machine for working with AI, which is where I do most of the testing.

I aggressively try to keep costs down so the workflow DSL I have supports configurable limits which can be set at the $, token, or time dimension , at task, workflow and agent/node levels, with some same defaults. I have a pipeline which keeps LLM API pricing data up-to-date, and I use AI to estimate total costs before runs and manually approve those.