alt Hacker News60% Fable cost cut by converting code to images and having the model OCR it
Comments
I tried the same thing last year (with openai models), back then it worked to reduce prompt tokens, but you needed way more completion tokens, ultimately more expensive (and slower) https://pagewatch.ai/blog/post/llm-text-as-image-tokens/
This seems like a pricing hack that burns resources, that when the loophole gets closed the price of OCR will have to rise?
I think Oh-My-Pi (OMP.sh) uses images for context compactificaton. OMP is built on top of Pi coding agent.
Tangentially related: I don't think OCR is the right term and I am generally vocal about that. But seeing this unquestioned here, I am wondering if I am the one who is wrong here. Is it ok to call this OCR? To me ocr means text in the end, not visual tokens.
Saw a Tweet a while ago from someone (maybe Carmack, maybe Geohot, maybe Karpathy?) wondering if images were just the better option.
Since then I've been using images with very simply worded prompts whenever I'm informing an agent of what is happening. Sometimes no text in the prompt at all.
It has been very very effective.
That being said, this isn't really what Karpathy was talking about. But it got me thinking a bit, and that got me to a much nicer workflow.
there's also a DeepSeek whitepaper on this technique https://www.seangoedecke.com/text-tokens-as-image-tokens
seems really dumb and like it would need to violate basic information theory to work?
input tokens are cheaper than output tokens. seems like it would maybe reduce input tokens at the expense of many more output tokens if you're actually triggering OCR via thinking?
Interesting approach, though that readme really needs a rewrite by a human...
Binary compression unpacked by OCR? This is the stuff of nightmares. So cursed, and yet...
I'm sorry, but this is retarded. It works, and it's clever, but but it's clearly a workaround for a pricing failure. Much like the bounty on poisonous snakes leading to people taking up snake-breeding, this just exploits and promotes waste. I think ultimately blame falls on Anthropic for the poor pricing system the enables such arbitrage. But I'm also disgusted by the inevitable tide of people exploiting this until its fixed, and creating an entirely unnecessary extra tide of digital junk.
Are we really re-discovering that compressed binary formats are more efficient data representations?
Reminds me of caveman: https://news.ycombinator.com/item?id=47647455
This probably works with PDF parsing as well I’m sure, even if it’s just from not having to parse pdf format alone.
What about: "Read this document online : [URL]" and you add your text/context to an online document?
Would that reduce the number of tokens used too?
That is hilarious and an amazing find.
a pictures worth a thousand tokens
a picture paints a thousand words
Isn't this basically what DeepSeek came up with https://github.com/deepseek-ai/DeepSeek-OCR
No words.
I cant get past that LLM intense slop text in the Github repo
[dead]
[flagged]
[flagged]
[flagged]
In Gemini at least, if you look at how they process PDFs, they do an OCR and then feed the text + image to the model, without charging you for the text tokens (I believe).
So my guess is that Claude’s backend is doing the same — so this hack is probably more of a loophole in token accounting that might get closed if Claude is doing what Gemini does