alt Hacker NewsVomit: Clean up Claude 5's token output with a separate LLM
Comments
At some point one has to wonder if it's still worth using anthropic's models if we need to babysit 100% of its output with another vendor's model. Why not just use that other vendor's model for everything?
I can't help but feel the circumstances that enable this kind of front page article are vestigial from the days when OAI was super bad and Anthropic was beyond reproach. This change-over-time is why I avoid getting tribal with technology vendors. Assigning ideological motives to 200k+ employee organizations is how we wind up in weird contortions like this.
Most rational actors simply moved from one to the other. It takes a special kind of devotion to the proverbial hole in the ground to keep pushing in this direction.
Looks like a wrapper around this prompt:
You are an editor. You'll be given a message with strange characteristics:
- Weird subject and verb combinations
- Subjects that should be objects
- Very roundabout reasoning, peppered with pseudo-epiphanies
- A distracting beat to the flow of the message
- Self-praise
Remove these characteristics, and rewrite it in a clear, conversational style. Keep the intent of the message, and take care not to lose any of the details.
A few specific rules:
- The message is usually set in the first person
- Only humans, groups of humans, and agents should do "action verbs"
- Objects should never do anything. Here are some examples to avoid:
- X carries ...
- X names ... - APIs are a minor exception to the action verb rule. They can do stereotypical things like CRUD, queueing, running, and calling.
- Avoid em dashes (—), as adds a distracting beat
The whole message you get is one block of that output. Reply with the edited prose and nothing else.
I'm not sure I want another layer of indirection personally, and I'm guessing an updated Claude model will reign this in at some point. I have however created a skill I call "deslop" and I invoke it to clean up Claude output when it goes off the rails. Here's the skill if anybody is curious:
https://gist.github.com/bmurphy1976/47ad81a842ab4b1628ef5974...
A small preview:
*Meta commentary.* Sentences about the document, the diagram, the reader, or the
writing itself ("the split across this diagram is the whole point", "a reader who
assumes X will be wrong", "as we'll see below"). Delete the frame and keep the fact
it was wrapped around. If there is no fact underneath, delete the sentence.Meta to this is anyone remember those days - ages ago now, probably months at least! - when Anthropic’s moral stance against the administration (combined with general consensus they had by far the best model) was making them the underdog champion that got a swell of support on HN? Recently the temp on HN seems to be that they’ve jumped the shark? Their brand doesn’t ooze ethics any more and their models disappoint?
I suspect that sustained reading of Opus 5's unconscionably bad prose could actually cause psychological harm. We're strongly considering moving all of our Anthropic spend to Codex/open weight models. It's a mental health decision at this point.
Apposite name but — grim trivia — bear in mind some emetophobes have a meaningful physiological reaction to the word and various euphemisms.
I have tried not to use it analogously ever since someone pointed this out. The word itself causes discomfort for a lot of people, many of whom will be surprised by it out of context, but in a non-trivial fraction it causes proper discomfort.
If you want people to use your tool it is probably better not to invoke nausea with its name or commands.
Just set the following incantation:
You must use ASD-STE100 Simplified Technical English (STE) when it doesn't detract from meaning.Along similar lines, a coworker told me about Caveman and it has made my Claude experience vastly better. I'm already a programmer, Claude shouldn't need to tell me like I'm 5.
Claude's verbosity is the biggest obstacle in the technical parts of my job currently.
(Why does this project need so much Go code to pipe something through a local LLM?)
Which Claude 5? Opus 5 does seem to have diarrhea of the mouth. But Fable 5 hasn't been so bad for me. Or perhaps it is just better at adhering to my guidelines.
I hope at some point Anthropic does a post-mortem on the strange behavior their models have been displaying recently. I mostly switched to Codex because I was finding Claude's behavior increasingly frustrating.
Very interesting you identified “carries” as well. I have been working on a claude.md to effectively ban this as well as forms of “hold”, “spells”, “sitting”, using “where” instead of “when” (except in SQL), and “pins” other than when pinning an assumption or version of something. This has helped a bit, but Opus 5’s prose is really quite bad.
The linked blog post has an example at the end. I think it made it much worse.
Telling Opus 5 or Fable to route answers through Opus 4.6 usually works pretty well. 4.7 really was the version where the writing style became horrible. I also have a Codex subscription in addition to to Claude 20x, that i use mainly for rewrites of Claude doc vonit and explaining Claude's plans
I "downgraded" to Opus 4.6 which is the last one that didn't have these problems.
For the local folks, I found Muse Glimmer 30B to be great at writing good technical stuff. It has good enough comprehension that it can take in a repo and find the relevant stuff that I ask for, and the output style is a breath of fresh air, with no fluff, ootb.
I think this is just another part of the growing pains of working with machine intelligence that we have to endure.
Much like we previously had to cope with "hallucinations" as an issue.
If the ultimate goal of AI is to develop general intelligence, the first big objective is: thinking systematically. And the road toward systematic thinking right now is mainly coding, mathematics, and other "verifiable reward" domains.
Claude doesn't have a separate mind for "coding" and "writing". Claude has tokens, and tokens can be assembled in various productive structures, mainly optimized right now for systematic reasoning. Also, a token isn't just a chunk of text. A token is like a little neural-network subroutine that fulfills a function. The conversion of a token into a piece of text only happens on the output side...
When the model finds token sequences that lead toward better verifiable outcomes, it leans hard into those token sequences, and uses them as an essential component of its thought process. "Load bearing" is load-bearing. "Verify, rather than assume" is a mantra that produces good results, so it gets repeated over and over again.
It's super-interesting that this particular moment, where the idea of "Claudish" has become a full-fledged meme, coincides with such astonishing progress in coding and math. My wife says when she uses Claude, that it feels to her exactly like talking to an autistic Engineer.
Not a coincidence, I think :)
My feeling is that the next big era of machine intelligence will require more lateral-thinking and creativity, and hopefully then the models "writing" will be more pleasant to read.
I have very little in my agents.md file wrt style, except for:
> Caveats belong inline, no "one thing to note" or "it's worth mentioning" footer. If it is worth raising or calling out, do so where it is most relevant and not as a foot note.
Opus 5 has a god awful habit of always doing a Columbo on every single response, and it is such a jarring read that it amps my cognitive burden having to back-read everything.
Ahh! Author here. Thanks for posting! In case anyone wants to share in my frustration, I also wrote a blog post about this.
Blog post: https://zachahn.com/posts/1787191554
The prompt I use to tell the LLM what to fix: https://github.com/zachahn/vomit/blob/main/internal/config/s...
Wasn't received too well on Lobsters haha, wrote a small extra blurb about it there: https://lobste.rs/s/juekuk/how_fix_claude_5_s_token_vomit
You think it's bad in English, but it's even weirder in other languages.
I have the same problem and I wondered what if the chatty output is a requirement for its (better?) reasoning capabilities?
The better approach is to stop the LLM in its tracks the moment it emits jargon or tortured metaphor and inject a turn that tells it what's expected instead.
The joy of watching a dumb AI-ism be sharply corrected by code you wrote months ago is hard to explain.
I'm not understanding the issue here, as everyone has different expectations for prose. No matter which LLM I use, I will provide a skill to generate high-quality prose. It is the first thing I add to my repo that I'm working on.
Or just take full control of your agentic coding experience with Pi Coding Agent and picking and choosing your favorite model's API discounted on flex pricing on deepinfra.com instead.
I highly recommend it.
Claude and Codex usage limits cannot be trusted.
Paying your own API bills in full is superior.
Just switch back opus 4.8, it's just as capable and you can actually understand the output
„Contributors: Claude“
Concise output mode only helps a little bit. Tools like this still have a reason to exist.
Regex pre-filter. You're welcome.
re.sub(r'(?is)\b(?:honest|caveat|absolutely right)\b.*', '', text)I’ve been using the pattern of using coding agents to orchestrate my CLI agents and it’s really good for these kinds of things
The vomit never makes it my way
From the README.md
> Anything that uses the OpenAI API?
I would have thought they meant the Anthropic API or maybe I'm misunderstanding?
I'm surprised by this reaction to Claude's verbiage recently. I don't have any issue immediately understanding what it's saying, but then again I read regularly and a lot of the people I know complaining think it's an accomplishment in literacy to get through Dungeon Crawler Carl.
There are a variety of political tensions in the US associated with whether academia has its head up it's ass (a right leaning perspective), or whether it's populated by experts that need to be supported and listened to (a left leaning perspective).
There's an echo of that tension in OpenAI vs Anthropic. For a while OpenAI seemed reckless and ignorant, preferring to just throw compute at the problem. Meanwhile Anthropic is hiring philosophers. But now that Claude has its head up its ass to the point where nobody wants to talk to it, OpenAI is looking rather pragmatic.
It brings to mind a skepticism about just letting the ivory tower do its thing without some kind of anchor to the everyman (this is why we make researchers also be teachers, though I'm not sure what the AI equivalent of that practice would be).
Watching the models seesaw in the same ways that humans do, but faster, is so surreal. I wonder if their tendencies will remain an echo of ours, or if they'll one day be more of a forward projection, a representation of where were going if we don't change our ways, and if we're lucky, a reason to change them.
This framing is spot on
With a whole separate LLM? FWIW you can also consider using something deterministic like vale.sh [1] , it's pretty easy to write prose violation plugins [2]
[1] https://vale.sh/
You don’t have to be so convincing when it’s a local model.
```Un-Claude 0.2beta
import sys,csv,requests
CH="# Valid channels: analysis, commentary, final. Channel must be included for every message."
CANDIDATES=[
("no-hedging","Reasoning: low\n\n<terse><no-hedging>\n\n"+CH,"Condensed:"),
("neutral-reg","Reasoning: low\n\nRegister: neutral technical. No intensifiers, no evaluative adjectives.\n\n"+CH,"Condensed:"),
("no-closing","Reasoning: low\n\n<terse>\nNo closing remarks.\n\n"+CH,"Condensed:"),
("terse","Reasoning: low\n\n<terse>\n\n"+CH,"Condensed:"),
]
def rephrase(text,base="http://127.0.0.1:1234",model=None,temperature=0.0,max_tokens=1400,timeout=180):
src=text.strip()
if not src:
return []
if model is None:
model=requests.get(base+"/v1/models",timeout=timeout).json()["data"][0]["id"]
w=csv.writer(sys.stdout,lineterminator="\n")
w.writerow(["idx","label","prefill","src_chars","out_chars","ratio","tokens","finish"])
rows=[]
for i,(lab,sysmsg,pf) in enumerate(CANDIDATES,1):
p="<|start|>system<|message|>"+sysmsg+"<|end|><|start|>user<|message|>"+src+"<|end|><|start|>assistant<|channel|>final<|message|>"+pf
d=requests.post(base+"/v1/completions",json={"model":model,"prompt":p,"max_tokens":max_tokens,"temperature":temperature},timeout=timeout).json()
c=d["choices"][0]
t=(pf+c["text"]).rstrip()
w.writerow([i,lab,pf,len(src),len(t),round(len(t)/len(src),3),d["usage"]["completion_tokens"],c["finish_reason"]])
rows.append((i,lab,sysmsg,pf,t,d["usage"]["completion_tokens"],c["finish_reason"]))
print("\nmodel: %s"%model)
print("temperature: %s max_tokens: %s"%(temperature,max_tokens))
for i,lab,sysmsg,pf,t,tok,fr in rows:
print("\n[%d] %s"%(i,lab))
print(" system: %s"%sysmsg.replace("\n","\\n"))
print(" prefill: %r tokens=%d finish=%s"%(pf,tok,fr))
print(t)
return rows
```input
## 8. Honest gaps — what I could *not* resolve
I want to be explicit about the limits of this pass rather than imply completeness:
1. *`PROVIDER_T` values are not enumerated here.* `list_models(inference_provider=...)` is typed against `PROVIDER_T`, which lives outside the three modules I scanned (it's in the `inference._providers` subpackage). The accepted provider strings are therefore *unknown from this run* — `"cohere"` is confirmed only because it appears in a docstring example.
2. *Three grep hits point to search-capable functions I did not identify.* My scan found parameter assignments that don't belong to any function I enumerated: - line 3046–3050: `params["filter"]`, `params["sdk"]`, `params["includeNonRunning"] = True` — an additional Spaces-oriented endpoint with an *`sdk` filter and an `includeNonRunning` flag* not exposed by `list_spaces`. - line 2879: `params["config"] = config` - line 12013: `"sort": sort` — almost certainly the consumer of `DailyPapersSort_T`, i.e. a daily-papers lister distinct from `list_papers`. - line 13872: `params["search"] = search`
These represent **real additional search surface** that my `LIST_FUNCS` whitelist missed. A follow-up pass enumerating every `HfApi` method containing `params[` would close this.
```example output
[1] no-hedging system: Reasoning: low\n\n<terse><no-hedging>\n\n# Valid channels: analysis, commentary, final. Channel must be included for every message. prefill: 'Condensed:' tokens=131 finish=stop
Condensed:
- *Provider strings* (`PROVIDER_T`) are not listed; only “cohere” is known from a docstring.
- *Missing search‑capable calls* were found: - `params["filter"]`, `params["sdk"]`, `params["includeNonRunning"] = True` (Spaces endpoint). - `params["config"] = config`. - `params["sort"] = sort` (likely a daily‑papers lister). - `params["search"] = search`.
These were not captured in the `LIST_FUNCS` whitelist, indicating additional search functionality.
```
Edit: yeesh, I’d love to have a WYSIWYG comment block on this site. I’m not going to keep fighting newline and white space to get it to look right, but you get the idea.
Sounds like LLM centipede.
I'm sorry but the whining over LLM output styles is embarrassing. Do Claude and GPT models always respond in exactly the way my most articulate coworker would? No. The overused jargon is absolutely annoying. But these things aren't my drinking buddies, they're professional tools. It's not _literally unreadable_. It's just not ideal. Most of my tooling is "not ideal". That's okay. That's what I'm paid for. I just work around it.
For me I added some instructions to speak clearly and it helped marginally and that's fine. There will be a new model out in a few weeks where I'm sure they've laser focused on this issue since nobody can shut the fuck up about it. The same thing happened with GPT if anyone can recall the ancient period of 4-6 months ago.
yes.
[dead]
[flagged]
I've been grappling with this for weeks, not just in Claude but in Codex as well, which isn't quite as bad but still annoying. AGENTS.md does very little, agents will consistently violate the communication preferences, especially as the session drags on. It's incredible to me that there's no good way to reliably change the way an LLM responds to you that a workaround like this would even be necessary. It seems like such a failure to live up to the promises of the product.
The baked in communication style of these models is so obnoxious it's impacting my work. The best way I can describe it is that everything is optimized to impress the user and make the agent sound more authoritative, but the way this is done is through deliberate obfuscation, inserting inappropriate and extremely dense jargon, and bizarre, stilted metaphors. It's like they've been trained to produce output that's hard to read.