Snapshots, increments, and the book on your screen
Two displays of the same market, side by side, showing different quantities at the same price. Or a book that was correct for an hour and then quietly stopped matching the fills you were getting.
Neither is a bug in either display. The venue does not send anyone a book — it sends a starting position and then a stream of changes, and every screen is a separate reconstruction from that stream. The ways a reconstruction can drift from the engine’s actual state are finite, well understood, and completely invisible once they have happened.
Three levels of detail
Venues publish market data at different granularities, and the words for them are conventional rather than standardised.
Top of book. The best bid and best ask, with their quantities. The smallest useful feed, and enough to know the spread and nothing about depth.
Aggregated depth. Total quantity at each price level, for some number of levels either side. This is the familiar ladder. Individual orders are summed, so 3.0 units at a price could be one order or thirty, and there is no way to tell.
Order-by-order depth. Each resting order published separately, with its own identifier. This is the only granularity from which anything about queue structure can be reconstructed, and many venues do not offer it at all.
The important consequence: at aggregated granularity, your own resting order is inside a total and indistinguishable from everyone else’s. The feed does not label it.
Snapshot plus increments
Publishing the whole book on every change would be enormously wasteful, since almost nothing changes each time. So the standard design is two parts.
A snapshot is the complete state of the book at one moment. It is the only thing that can establish a starting position.
An increment — an update or delta — is a description of one change: quantity at a price became this value, or a level is now empty, or a trade occurred. Increments are small and frequent.
The reconstruction is therefore: take a snapshot, apply every increment after it in order, and you have the engine’s state as of the last increment applied. Every part of that sentence is load-bearing. Miss an increment and the book is wrong from that point on. Apply them out of order and it is wrong. Start from a snapshot that is older than the increments you have been buffering and you have a gap you cannot fill by waiting.
Sequence numbers and gaps
Because order and completeness are both essential, feeds carry a sequence number: a counter, incremented per message, that lets a receiver detect that something is missing.
Its whole purpose is negative. It does not fix anything; it tells you that your reconstruction has become invalid. Detecting a gap means the correct action is to discard the local book and start again from a fresh snapshot, because there is no way to infer the contents of a message that never arrived.
This is where silent failure lives. A reconstruction with an undetected gap continues to work — it applies subsequent increments happily, produces a book that looks entirely normal, and is simply wrong at whichever levels the missed message touched. Nothing about the display suggests a problem. The first evidence is usually a fill that does not match the depth you were reading, which arrives long after the gap.
Conflation, throttling and depth limits
Even a perfect reconstruction is not the engine’s book, because what is published is frequently less than everything that happened.
Conflation replaces several changes at a level with one message carrying the final value. Nothing published is false, and the intermediate states are gone — so a level that was consumed and refilled between publications appears never to have changed.
Throttling limits messages per unit of time, which does the same thing by a different route: you receive the state at intervals rather than the sequence of transitions.
Depth truncation. A feed carrying some number of levels tells you nothing about the ones beyond it. An order large enough to reach past the published depth is priced against a book you have not seen the relevant part of.
Each of these is a deliberate design choice with a purpose, and each removes information in a direction that makes the market look calmer and thinner-changing than it was.
The mechanism
THE MECHANISM — from engine to screen
· Venue publishes
→ a snapshot plus a stream of
increments. Never a book.
· Increments applied in order from a
snapshot
→ a correct reconstruction, as of
the last message applied.
· One increment missed
→ book WRONG from that point on,
permanently, and it still looks
normal.
· Sequence gap detected
→ the only repair is a fresh
snapshot. Nothing can infer the
missing message.
· Conflated or throttled feed
→ intermediate states never
published. Consumed-and-refilled
levels look untouched.
· The book you are reading
→ NO GUARANTEE of currency. It is a
past state of the engine, always.
· Granularity offered, snapshot cadence,
conflation intervals and depth limits
→ VENUE-SPECIFIC. Aggregated and
order-by-order feeds of the same
book disagree in what they can
show.Worked example
Illustrative figures, synthetic throughout. Suppose a bid level at 40,000 and this sequence of events at the engine, in order.
Message 11: 40,000 has 5.0 units. Message 12: 40,000 has 2.0 units — 3.0 was consumed. Message 13: 40,000 has 6.0 units — 4.0 was newly posted. Message 14: 40,000 has 6.0 units and the level above it changed.
A receiver that gets all four reconstructs 6.0 at 40,000 and knows the level was consumed down to 2.0 along the way.
A receiver on a conflated feed may get one message covering the whole window: 40,000 has 6.0 units. The final state is identical and correct. The fact that the level was briefly thin is simply absent — and anyone measuring resiliency from that feed cannot see the event that resiliency is about.
Now suppose message 12 is lost and the gap is not detected. The receiver applies 13 and 14, so the level reads 6.0, which is right by coincidence — the messages carry absolute quantities and the later one overwrote the error. Change the design so increments carry deltas rather than absolute values, and the same loss leaves the level permanently 3.0 too high, on a book that continues to look plausible indefinitely.
That difference — absolute values versus deltas — decides whether a missed message self-heals or poisons the reconstruction, and it is a venue design detail rather than anything you control.
Why two screens disagree
Given all of the above, two correct displays of one market differ for at least five reasons, none of which is an error: different granularity, different depth limits, different conflation, different snapshot timing, and different network paths.
Add a sixth for aggregated displays: rounding and level grouping. A display that groups prices into buckets coarser than the venue’s tick is showing sums across several real levels, and an order priced inside a bucket is priced against a level the display never distinguished.
The reconciliation people want — one true book — does not exist outside the engine, and the engine does not publish itself.
The failure mode
Every reconstruction is stale, and staleness is the benign part. The dangerous part is that a reconstruction has no way to signal how stale or how wrong it currently is.
The specific way it bites: an order priced against a level that no longer exists behaves differently from what you intended — it crosses further than expected, or rests somewhere you did not mean, or is rejected against a reference you were not seeing. The instruction was correct with respect to your book and your book was not the engine’s.
There is no fix, only an accurate description: you are always acting on a past state, the gap is sometimes larger than usual, and it is never zero. Every other mechanism on this site sits downstream of that, and the honest form of any depth-based estimate carries it as an assumption rather than pretending it away.