Coinbase Sliding Window

A command-line application that subscribes to the Coinbase Exchange ticker channel and, for every trade, computes the rolling mean, median, low and high of the trade price over a sliding time window. Each trade becomes one CSV row per instrument.

Where the Gemini feed handler is about where the time goes, this one is about getting four numbers exactly right. Coinbase sends a few trades a second per instrument over the public internet, so nothing here is latency-bound. The interesting part is keeping each statistic correct as trades enter and leave the window, without the memory growing with the window.

A live run

Three instruments at a 5-minute window, about 11 minutes of trading on 10 September 2026. The shaded band is the window’s low and high. Left of the dashed line the window is still filling, so every statistic starts at the first price and spreads out as trades accumulate.

The charts are drawn from the application’s CSV output, unmodified. Each row is one trade, with the window’s statistics as of that trade:

time,trade_id,price,mean,median,low,high
2026-09-10T15:28:16.731415Z,1091155129,77078.78,77078.77,77078.77,77078.77,77078.78

Architecture

connect -> subscribe -> producer: read -> parse JSON -> tick -> SPSC enqueue
                        consumer: dequeue -> window update -> CSV

Two threads joined by a lock-free single-producer single-consumer queue. The producer owns the socket and a simdjson parser, and turns each ticker message into a 32-byte tick: timestamp, price, trade id and instrument index. The consumer owns one window and one CSV writer per instrument. The statistics update is a handful of integer operations, so it stays inline on the consumer. A second queue in front of the CSV writer would cost more than it saves.

At a few trades a second, an SPSC queue is more than this feed needs. I built it that way because it keeps network ingest apart from computation, which is the shape a real feed handler has.

Prices are integers. Each price is parsed from its JSON string straight into an int64_t at 10^-6 scale, so no floating point touches the price path. The CSV writer converts back and trims trailing zeros.

The exchange’s clock drives the window. Window edges come from the timestamp in each ticker message, not local receive time, so replaying the same input gives identical output. Timestamps become epoch nanoseconds through Howard Hinnant’s days_from_civil, and go back through civil_from_days for output.

Four statistics, four structures

The window is a ring buffer over a flat array with a power-of-two capacity, so wrapping is a mask. Each push takes the right edge from the newest timestamp and expires ticks from the front. When the ring fills, it doubles up to a ceiling, and past that a tick is refused rather than overwriting unread data. An occasional reallocation stops once the buffer settles; a dropped tick would corrupt every statistic for the rest of the window.

Mean. A running sum, incremented on push and decremented on expiry, divided by the occupancy. It is the one statistic that is not a traded price, so it is truncated to two decimal places to match the other columns and can read up to a cent below the true value.

Low and high. Two monotonic deques. When a trade arrives, every smaller value behind it in the max deque is dropped, because it is shadowed by a larger value that also expires later and can never be the maximum again. The min deque mirrors this. The front of each deque is the answer.

Median. An exact sliding median needs every price in the window held, so its memory grows with the window. At an hour of trading that stops being reasonable. Instead the median uses a fixed-size count histogram over the price domain: a push increments a bucket and an expiry decrements one, both O(1) whatever the window length. Reading the median walks the buckets from the low end, accumulating counts until the total passes n/2. The walk is bounded by the low and high from the deques, so it costs the observed price range rather than the whole array.

Bucket width is TICK_SIZE cents, and it alone sets the resolution:

A coarser bucket reports its lower edge, so the error is one-sided. For an even count the upper of the two middle values is reported rather than their average, so the median is always a price that actually traded. ETH and SOL run at 1 cent. BTC runs at 10 cents, which keeps its histogram at 200,000 buckets across a ±$10,000 range.

Testing

Tests use Catch2 and run on every push through a Gitea Actions workflow. The test vectors are generated in Python with datetime and timedelta integer arithmetic:

• 1,000,000 timestamps across several centuries, checked in both directions
• 1,000,000 prices with zero to six fractional digits, including sub-$1 values and whole numbers, checked in both directions
• two generated feeds, 10,000 trades over a 1-minute window and 1,000,000 over a 1-hour window, checking occupancy, head, tail and exactly which trades survive expiry, trade by trade

The statistics themselves are covered by two 20-trade fixtures with expected values worked out independently of the code. One runs at a 5-minute window for the accumulate path. The other runs at 1 second for expiry: several trades expiring at once, the low and high ageing out, duplicate timestamps, and a trade sitting exactly on the window boundary. Duplicate timestamps are common on the live feed, since a trade that fills against several resting orders arrives as several ticks with the same time.

A few things I had to debug along the way

The first came from the live feed. The rest showed up as failed assertions against the generated test vectors rather than as crashes.

A malformed subscribe message. Coinbase answered the first subscription with Malformed JSON. The separator after the last instrument was the character '\0', and std::string stores a NUL as a real character, so the message went out with bytes that were not JSON.

Timestamps off by a microsecond. 17 of 1,001 timestamp assertions failed, all far in the past and all off by exactly 1,000 ns. The C++ was right. Python’s datetime.timestamp() returns a float, which cannot hold nanoseconds that far from the epoch. Building the expected values from timedelta’s integer fields fixed the generator.

Garbage when formatting dates before 1970. Converting nanoseconds back to a string produced output such as 1788-05-26T�5:�7:�6.�75643Z for negative values, because C++ integer division truncates toward zero rather than rounding down. The fix borrows one from the next unit up whenever the remainder comes out negative.

Trailing zeros, twice. The price round trip failed on values such as 65852.80 against 65852.8. The trimming logic did not stop at the six-digit fraction, and the Python generator was not trimming at all. Once both were fixed, a larger run of 218,883 assertions found exactly one more failure: 0.53 printed as .53.

A median index that wrapped. The 1-hour feed hit an assertion with a bucket index just under 2⁶⁴. The bucket-width variable was a size_t, so the whole index calculation was done unsigned and a negative result wrapped. The same run also showed the generated prices wandering further from the first price than the histogram covers, which is the rebase limitation below.

Known gaps

• The median histogram never rebases, and its index is not bounds-checked. It is anchored to the first price seen. A trade more than MASK cents from that anchor falls outside the array and writes out of range. The fix is to recentre and rebuild when a price escapes, which is O(window), rare, and cheap to trigger because the deques already hold the new bounds. Not implemented.
• No clean shutdown. The producer blocks in the socket read and there is no signal handling, so the process stops on Ctrl-C and skips the CSV writers’ final flush. Up to 2 KB of buffered rows per instrument can be lost.
• A malformed price parses as 0 instead of being rejected.
• A full queue drops the tick with a log line rather than applying back-pressure.
• Linux and GCC only. Verified on Fedora (Asahi, aarch64) and Ubuntu (x86-64). The code does not yet compile under Clang, so macOS is unsupported.
• The timestamp parser reads fixed offsets. It assumes zero-padded fields, microsecond precision and a Z suffix, which is what Coinbase sends, and does not handle numeric timezone offsets.

Trade-offs and things deliberately scoped out

The histogram is sized per instrument, by hand. MASK sets how far either side of the first price the histogram reaches, and TICK_SIZE sets the bucket width, so memory is 2 × MASK / TICK_SIZE buckets. Because the range is centred on the first price, MASK cannot exceed that price in cents, or every tick for the instrument is refused. Both are command-line arguments per symbol, chosen for current price levels:

An instrument that repriced by an order of magnitude would need new values.

The consumer busy-spins. It polls the queue rather than sleeping on it, which holds a core for a feed that delivers a few trades a second.

CSV output stays on the consumer. The writer buffers and flushes in blocks, so it is cheap enough to run inline. A second SPSC handoff would only pay off if the write ever became synchronous.

One connection, blocking receive. The WebSocket client reads with a blocking call on the producer thread, which suits a single connection to a single exchange and is also why there is no clean way to interrupt it at shutdown.

Where I had help

I used an AI assistant as a research aid. It explained the mechanics behind fixed-point price parsing, pointed me to Hinnant’s date algorithms after I turned down timegm() as too slow on the hot path and a cached-date approach as error-prone around midnight, and suggested Catch2 rather than writing a test harness of my own. When I was stuck on two of the test failures above, it identified the Python float and the truncating division.

Source

The repository lives on a self-hosted Gitea instance and is not publicly accessible. A read-only snapshot of the current code, synced at build time, is browsable here:

Browse the source

The WebSocket client this builds on is my own, written on top of Boost.Beast for the Gemini project and adapted here. It handles the connection, TLS handshake, subscription and blocking receive, and is included in the snapshot under dep/websockets/. The vendored third-party libraries next to it (simdjson and Catch2) are not.