logoalt Hacker News

rednafitoday at 8:52 AM4 repliesview on HN

I dream of a SQL like engine for distributed systems where you can declaratively say "svc A uses the results of B & C where C depends on D."

Then the engine would find the best way to resolve the graph and fetch the results. You could still add your imperative logic on top of the fetched results, but you don't concern yourself with the minutiae of resilience patterns and how to traverse the dependency graph.


Replies

burakemirtoday at 12:40 PM

You could build something like this using Mangle datalog. The go implementation supports extension predicates that you can use for "federated querying", with filter pushdown. Or you could model your dependency graph and then query the paths and do something custom.

You could also build a fancier federated querying system that combines the two, taking a Mangle query and the analyzing and rewriting it. For that you're on your own though - I prefer developers hand-crafting something that fits their needs to a big convoluted framework that tries to be all things to all people.

Garleftoday at 10:09 AM

Isn't this a common architecture in CQRS systems?

Commands to go specific microservices with local state persisted in a small DB; queries go to a global aggregation system.

torginustoday at 12:07 PM

I think SQL alone is great if you didn't drink the microservice kool-aid. You can model dependencies between pieces of data, and the engine will enforce them (and the resulting correct code will probably be faster than what you could do otherwise).

Then you can run A,B,C and D from a consistent snapshot of data and get correct results.

The only thing microservices allow you to do if scale stateless compute, which is (architecturally) trivial to scale without microservices.

I do not believe there has been any serious server app that has had a better solution to data consistency than SQL.

All these 'webscale' solutions I've seen basically throw out all the consistency guarantees of SQL for speed. But once you need to make sure that different pieces of data are actually consistent, then you're basicallly forced to reimplement transactions, joins, locks etc.

kgwxdtoday at 10:31 AM

Datomic?