logoalt Hacker News

WatchDogtoday at 12:30 AM1 replyview on HN

Token usage is a different and more sympathetic heuristic than LOC produced.

The metric by itself tells you nothing about what value those tokens produced, but to some extent it represents the amount of thinking you are able to offload to the computer for you.

Wide breadth problems seem to scale well with usage, like scanning millions of LOC of code for vulnerabilities, such as the recent claude mythos results.


Replies

HarHarVeryFunnytoday at 12:40 AM

The trouble with rewarding token usage is the same as rewarding LOC written/generated - if that's what you are asking for then that is what you will get. Asking the AI to "scan the entire codebase for vulnerabilities" would certainly be a good way to climb the leaderboard!

show 1 reply