Thank you for sharing!

I can appreciate surpassing foundation models in RelBench, but I’ve found that peers in our space strongly prefer RelArena because it reruns submitted implementations under shared evaluation protocols. IMHO Prior Labs does a great job of providing a controlled and transparent environment.

In RelBench, hypothetically, a solution could mine for days, weeks, or months for specialized, handcrafted features. I’d would be extremely interested in how the cumulative cost of that exhaustive search scales to new, enterprise-scale databases.

As per the leaderboards you reference: "The RelBench leaderboard uses self-reported prediction files, so we rely on the authors to submit in good faith ... For a more controlled and transparent evaluation on RelBench tasks, please see RelArena".

I would love to see Kapso’s performance in a controlled environment like RelArena, to understand how this approach compares with Relflow and other relational learning methods in an apples-to-apples comparison!