logoalt Hacker News

adamkurkiewiczyesterday at 7:48 PM1 replyview on HN

Hey, the author of the benchmark here.

The benchmark data was prepared in April 2026 (when I was manually doing our VAT return with my co-founder). The invoices were indeed found manually.

Currently we're using a custom "invoice searcher" built on Kimi 2.6 (in our testing several weeks ago it outperformed Opus 4.7; it was just more persistent).

Ultimately, I still verify everything manually after the model is finished fetching invoices for the month -- but it's a great help to have all the invoices already found (usually correctly).


Replies

breadisloveyesterday at 8:34 PM

adam, i'd like to get in touch and would love to run the benachmark with mixedbread as a search backend. we are doing this right now with a lot of compliance companies. would be very curious how it improves quality/cost e2e

show 1 reply