alt Hacker NewsCouldn't we make references as `byte offsets in the result set` work to handle duplication? Real memory pointers wouldn't work over the network of course, but if the database driver would return results like this, the client could easily stitch these together. My hunch is that even if we implement references on a higher level than raw byte offsets it would still be more performant than just returning R1*R2 bytes for any R1<1:N>R2.
---
EDIT: According to LLM friends you could achieve before wire de-duplication by using FOR JSON AUTO in SQL Server or jsonb_agg in PostgreSQL. Not sure how much overhead that incurs though.
Replies
You'd still return a multiplicative amount of rows, even if those rows contained only a reference. `array_agg` in postgres avoids this, but EF does not support using it for collection navigations.
One could envision a "Cartesian product" operation in the wire protocol, but I'm not convinced that's a good approach.
I think jsonb_agg could be a solution. Requires cpu for decoding but I think it is compensated by the reduction of on the wire traffic. Queries are different , especially if you have several one-to-many joins so you have to do it at the framework level. I wonder if it would be feasible a Postgres extension that does this automatically for you