logoalt Hacker News

phildenhoffyesterday at 7:22 PM2 repliesview on HN

The company I work for, Digits, has been regularly updating our AI-vs-human bookkeeper benchmark. Look at page 8 -- many models are nearly as accurate as a human bookkeeper

https://digits.com/downloads/beyond-the-hype-evaluating-llms...


Replies

ivababukovayesterday at 7:37 PM

Thanks, that's useful! We did use ChatGPT 5.5 for some time and it did perform pretty well too (of course more expensive than GLM 5.2). We tried Claude 4.7 and 4.8, but we found both models to be "lazy" and very expensive. Claude would always rather prefer the route of saying that the evidence was not found or something is incomplete, rather than put more effort into finding/repairing the particular issue.

senordevnycyesterday at 11:30 PM

I use Digits for my solo saas and it’s been great so far!