HarnessTax: How Much Does the Harness Matter for Coding Agents?

My own findings are in line with this research:

Having a coding harness is critical but the differences between them are overstated.

Personally, I’ve replaced OpenCode with a thin wrapper around Pydantic-AI as the pythonic analogue to Pi-Agent for headless use via Hermes

They’d all do the job - I just prefer to compartmentalize for access control.

Keeping the harness’ surface area tiny had the added benefit of preserving my understanding and being able to adapt it to my preferred workflow effortlessly

The term "harness" here is being overloaded for the term "agent", which is worrying. Putting that aside, there are many factors that matter. The "harness" context, the execution pattern (parallel vs sequential), the ability to delegate to other models, etc.

Optimal harnesses use concurrent execution + subagents and are not stuck on one model. Cost and performance are impacted GREATLY by these tactics, regardless of the native agent context (instruction). This kind of single-harness analysis is shallow and misleading, although the finding that "Provider-specific optimization does not guarantee the best pairing" is probably correct, depending on how you measure.

It is a starting point.

Say I'm using Claude Code or GPT Codex's harnesses but also sending some queries to the respective Anthropic and OpenAI models via OpenRouter.

Do harnesses and therefore sending the queries directly to the LLM providers have caching and other benefits that OpenRouter does not provide? Would I get any of those benefits if I simply proxied any requests to the major providers' harnesses through OpenRouter? Or only if the requests go straight from the harness to the provider's API?

Claude Code/Codex charge the user for their extremely bloated one-size-fits-all system prompts (including safety instructions and other stuff users dont want).

In my experience if you're using OpenAI/Claude models and paying API costs, almost every other harness beats Claude Code/Codex in cost.

If this is mostly because of the size of the system prompt, then perhaps in long horizon tasks the "tax" will be less obvious.

Does this extend to open models like GLM 5.3? This would mean that simply changing the harness to Pi reduces cost in half?

As the model gets smarter, you need to tell it less

Much of the extra weight of Claude Code and Codex harness are (ostensibly?) for security and alignment purposes. Whether they are effective is an open question, but leaving those dimensions out and calling it a tax is disingenuous, just turning insecurity into a negative externality.

"Why pay the waste disposal tax? Dumping into the ocean is free!"

Pi actively omits any sort of guardrails and sandboxing in the name of speed and simplicity, so it is not shocking that it is faster and simpler.

Doubling the cost of something in the name of vague security is standard operating procedure for big enterprises, maybe even quite cheap.

claude code feels mildly shitty to use in the way that every other vibe-coded-project-got-out-of-hand project does, which is like, not that bad, but it's fucking ridiculous for a 2 trillion dollar company's main companion product

[dead]