logoalt Hacker News

toshyesterday at 5:36 PM1 replyview on HN

I built an ad-hoc custom comparison framework to inspect system prompts, caching behaviour, tool call outputs, exact api requests and responses and so on

I agree fewer tokens is not necessarily better but a bit counter-intuitively often the harness using fewer tokens is not only done faster but has better results

(that said: of course check the results, look at the full traces, agree!)


Replies

azuanrbyesterday at 5:44 PM

Thanks! I ended up built my own too. Just thought there are other better options out there that I might've missed.