logoalt Hacker News

artinetoday at 5:55 PM0 repliesview on HN

I gave Claude Code with Sonnet 4.6* a try a few weeks ago. I pointed it at a hobby project with less than 1kloc of C (about 26,500 characters) across ~10 modules and asked it to summarize what the project does. It used about $0.50 worth of tokens and gave a summary that was part spot on and part hallucinated. I then asked it how to solve a simple bug with an easy solution. It identified the right place to make the fix but its entire suggested solution was a one-liner invoking a hallucinated library method.

I use LLMs pretty regularly, so I'm familiar with the kinds of tasks they work well on and where they fall flat. I'm sure I could get at least some utility from Claude Code if I had an unlimited budget, but the voracious appetite for tokens even on a trivially small project -- combined with a worse answer than a curated-context chatbot prompt -- makes its value proposition very dubious. For now, at least.

* I considered trying Opus, but the fundamental issue of it eating through tokens meant, for me, that even if it worked much better, the cost would dramatically outweigh the benefit.