alt Hacker NewsHi! I'm the creator of pgrust. Let me explain what happened.
For the most part, the actual accusations are false. This submission was an earlier version of pgrust 0.4 called pgrust 0.4-preview. We just open-sourced a newer version called pgrust 0.4-beta yesterday, and you can fully audit it here[0]. Rather than contest the accusations on GitHub, we figured it would be easier to complete pgrust 0.4 and open-source it to show that we're being honest. You can also verify what I'm saying yourself against the 0.4-preview binary we submitted to ClickBench here[1]. I don't have a way to prove this is the exact binary submitted, but if ClickHouse provided the SHA of the binary they tested against, it would match.
The three main accusations boil down to: 1) We used handwritten code optimized against each query in the benchmark, 2) We cached the results between query runs, and 3) We didn't support queries outside of the 43 ClickBench queries.
First, we did not use handwritten queries. The code Alexey pointed out does not run during the benchmark. The design of 0.4 relies on a JIT compiler that compiles your SQL to a Lisp-based intermediate representation that we call Gravity IR that we then convert to x86 and ARM assembly. While developing 0.4, I wrote handwritten code against the IR for benchmarking purposes. The idea was that if queries written in our IR are fast, and we can compile good enough IR fast, then our new query engine will be fast. This is also the reason the directory those "kernels" are in is called "irprobe" - it was a probe of our IR to make sure it was fast. Our 0.4-preview submission went through the pipeline: SQL query -> Gravity IR -> assembly.
Second, the "result-cache" Alexey observed is the result of a cache we use to cache which predicates match which strings in our dictionaries we use for dictionary compression. Alexey wasn't aware of this cache and assumed we were just caching the query results. On ClickBench you are allowed to use caches like this that hold intermediate data[2]. ClickHouse has its own cache similar to this called the Condition Cache[3]. In our newest submission, we were upfront about all the caching we do and how to disable each cache[4].
The third claim that we didn't support other queries outside of the 43 ClickBench queries is partially false, but does have some ground to it. As I said before, pgrust 0.4-preview was an early version of pgrust 0.4. Once our new JIT compiler supported enough SQL to handle the ClickBench queries, we decided to submit it to ClickBench. AFAIK, ClickBench has no rules about how much SQL you need to support. On top of that, we did ship that we did ship 0.4-preview with our old query engine. It would have been a two-line change to fall back to it and support a much broader set of queries outside what our new engine could support. Those queries would still be fast, but not quite as fast as our new engine was. Our new 0.4-beta release does now support all Postgres SQL (minus some bugs around CTEs and MERGE)
To sum up, pgrust 0.4-preview was an early build of pgrust 0.4 while it was in development. It was a real query engine and actually executed queries. I acknowledge that we submitted it prematurely, when it supported a narrow set of SQL, and we should have ensured broader SQL coverage before submitting it. With our new submission that solves these problems, and we're hoping for pgrust to be reinstated. If there's anything off about the submission, please point it out and we're happy to fix it.
[0] https://github.com/malisper/pgrust/releases/tag/v0.4-beta
[1] https://pgrust.com/downloads/archive/2026-09-clickbench-snap...
[2] https://github.com/ClickHouse/ClickBench/blob/main/README.md
[3] https://clickhouse.com/docs/concepts/features/performance/ca...
Replies
What's the explanation for paths to q00.rs-q36.rs being in the binary?
Even with your explanation, this whole thing seems weird and scammy. What's the point of submitting something that is so incomplete and heavily tailored to this one benchmark outside of self promotion?
Benchmarks should be used for measuring general purpose query engines, not be an end in themselves.
This seems like a prime example of Goodhart's law. This benchmark is now just being used as a target for self promotion.
thanks for your explanation. I'd love to see a concise explanation of the difference in benchmark performance between -preview and -beta
that would help clear the air, if the full-sql version has similar performance to the partial-sql version
Honestly, if the database is not able to be run with a variety of queries other than the one used by ClickBench then don't submit it to the benchmark. Every other database in their list can run anything you throw at it. Hiding behind messaging that it was an early 0.4 build that can't do everything is ... weird! Just don't submit it to ClickBench if it is not usable by the world yet.
I hope they don't allow pgrust in again until the project matured. Otherwise the benchmark will be useless as it gets flooded by many projects that can run ClickBench fast but have questionable support for everything else.
thanks for the explanation, let's first see if the binary matches.
> For the most part, the actual accusations are false. This submission was an earlier version of pgrust 0.4 called pgrust 0.4-preview.
This is a non-sequitur. It doesn't matter what the earlier version is called. It is what you submitted, and it is what the accusations are against. You don't get to create a new submission that (allegedly) fixes all of the issues and then say that the previous accusations are "false". Downplaying the issues by calling it a 'preview' when you submitted it yourself is also a bit of having your cake and eating it too, don't you think?
> ClickBench has no rules about how much SQL you need to support.
It is because of people like you that we have warnings not to eat tide pods, Jesus Christ.
Thanks for the explanation, can you release the source of 0.4-preview so that people can understand what was going on?