Strictly monotonic timestamps and gapless sequences in Rust

Something seems fishy here. What am I missing?

> no torn state

Doesn't there exist a failure mode where you allocate from a batch, fsync that you've done that allocation, the application fails to do anything with that counter, and you thereby introduce a gap in whichever "gapless" sequence you're trying to construct? The primitive exposed here might be able to be gapless, but the API doesn't look like it'll compose nicely into creating an application with the gapless property you're trying to enforce.

That desync between application state and tsoracle state seems more likely when you consider other features like requesting a batch of timestamps (which is necessary to hit the stated performance numbers).

> gapless, high-throughput, monotonic

WAT? You don't get all three of these at once with any implementation. Looking at this implementation, you (as predicted) don't get it here either. It basically uses a partitioned allocation strategy -- allocate a window to each node, each node allocates from its window (seems at first blush like a reasonably good implementation of that strategy, not knocking it, at least not without reading further). That's not both gapless and monotonic though. Absolutely nothing enforces within-window synchronization from one node to the next. You do (eventually) get gaplessness if no node is ever eventually always idle, but only because you explicitly left a hole to fill as you allocated a bigger number before sometimes filling that hole with a smaller number. It's not monotonic.

Edit 1: Okay, I see, the sequence interface is different from the timestamp interface. That resolves the "gapless, high-throughput, monotonic" concern. Not the other points yet.

Edit 2: Yeah, I have a lot of concerns about the timestamp itself re monotonicity. It looks like [physical_ms: u46; logical: u18]. There's basically no high-throughput way to use a structure like that to enforce monotonicity in any meaningful way across nodes. To illustrate the problem (not a proof, just intuition-building), zoom in right next to a physical ms boundary. Microseconds of skew can cause two machines right next to each other to issue timestamps on one side or the other of that boundary. No matter what you do with the logical partition, that noise is going to dominate the resulting sort order. If those two nodes publish their respective timestamps, you can absolutely see a later timestamp before you see an earlier timestamp. You could even have the slow node observe time t+1 from the fast node and then afterward produce time t. Maybe the devil's in the details with the definition of "monotonic," but that seems like an incredibly hard abstraction to use correctly. This is marketed as providing timestamps you can use to order events across machines, but I'm not seeing anything better than what you would get with just a per-node monotonic clock.

Do we know that this benefits from the same domain specific optimization that an organically made implementation would? Have you performed real-world comparative benchmarks? Stuff like this doesn't just come out of the blue with AI. Not saying you did a bad job, just skeptical

[flagged]

[flagged]

I'm probably being naive and stupid in ways I'm too stupid to imagine but isn't this basically just id = 0; GET() => id++

[flagged]

[flagged]