alt Hacker NewsI’ve been thinking that defaulting to durable execution over lower‑level primitives like queues makes sense a lot of the time, what do you think?
A lot of the "simple queue" use cases end up needing extra machinery like a transactional‑outbox pattern just to be reliable. Durable‑execution frameworks (DBOS/Temporal/etc.) give you retries, state, and consistency out of the box. Patterns like Sagas also tend to get stitched together on top of queues, but a DE workflow gives you the same guarantees with far less complexity.
The main tradeoff I can think of is latency: DE engines add overhead, so for very high throughput, huge fan‑out, or ultra‑low‑latency pipelines, a bare‑bones queue + custom consumers might still be better.
Curious where others draw the line between the two.
Replies
Highly biased opinion here since I'm the CEO of DBOS:
It'll be rare that the overhead actually has an effect, especially if you use a library like DBOS, which only adds a database write. You still have to write to and read from your queue, which is about as expensive as a database write/read.
Drawing the boundary at high throughput, huge fan-out and ultra-low-latency is correct - I'd also add that MQs are often used for pub/sub and signaling.
MQs are heavily optimized for reducing E2E latency between publishers and consumers in a way that DE engines are not, since DE engines usually rely on an ACID compliant database. Under load I've seen an order of magnitude difference in enqueue times (low single-digit milliseconds for the MQ p95 vs 10ms p95 for Postgres commit times). And AMQP has a number of routing features built-in (i.e. different exchange types) that you won't see in DE engines.
Another way to think about it is that message queues usually provide an optional message durability layer alongside signaling and pub/sub. So if you need a very simple queue with retries _and_ you need pub/sub, I'd be eyeing an MQ (or a DE execution engine that supports basic pub/sub, like Hatchet).
I wrote about our perspective on this here: https://hatchet.run/blog/durable-execution
( disclaimer - I'm one of the people behind https://github.com/hatchet-dev/hatchet )