# Coinbase Ticker Statistics
A C++17 application that subscribes to the Coinbase Exchange ticker feed and computes
the rolling mean, median and range (low/high) of the trade price over a sliding time
window, writing one CSV row per trade.
---
## Build and run
```bash
./build-all.sh
```
One script does everything: it installs the system dependencies (toolchain, OpenSSL,
Boost), builds the vendored dependencies under `dep/`, builds the feed handler and the
test binary, and generates the test vectors. It detects the package manager
(dnf, apt, pacman or brew).
Then either:
```bash
./run-fh.sh # live feed -> CSV
./run-test.sh # tests
```
`run-fh.sh` passes seven arguments:
```
<host> <port> <endpoint> <instruments> <MASK list> <TICK_SIZE list> <window size in ns>
```
It currently runs BTC-USD, ETH-USD and SOL-USD over a 5-minute window. Adding an
instrument means adding it to all three lists in that script.
**Linux and GCC only.** Verified on Fedora (Asahi, aarch64) and Ubuntu 26.04
(x86-64). The code does not currently compile under Clang, so macOS is unsupported.
The build scripts detect Homebrew, but the code itself has not been ported.
### Choosing MASK and TICK_SIZE
Both size the median histogram, and they are independent of each other.
`MASK` is given in cents and sets the price range the histogram can represent: `MASK`
cents either side of the first price seen. `TICK_SIZE` sets the bucket width in cents,
which controls resolution and memory but not range.
```
buckets = 2 x MASK / TICK_SIZE
bucket width = TICK_SIZE cents
range = first price +/- MASK cents
```
Because the histogram is centred on the first price, `MASK` must not exceed that price
in cents, or the lower edge would fall below zero. When it does, every tick for that
instrument is refused. `MASK` therefore scales with the instrument's price level, which
is why it is a per-symbol argument rather than a constant.
The values in `run-fh.sh`: ETH and SOL run at `TICK_SIZE = 1`, so their medians are
exact. BTC runs at 10, a 10-cent bucket on an instrument trading near $77,000, so its
median error is under $0.10. That keeps its bucket count in line with ETH despite the
much wider range:
| symbol | MASK | TICK_SIZE | range covered | buckets |
|---|---|---|---|---|
| BTC-USD | 1,000,000 | 10 | +/- $10,000 | 200,000 |
| ETH-USD | 100,000 | 1 | +/- $1,000 | 200,000 |
| SOL-USD | 5,000 | 1 | +/- $50 | 10,000 |
These assume current price levels and would need revisiting if an instrument repriced
by an order of magnitude. Raising `TICK_SIZE` cuts memory proportionally at the cost of
median resolution; see the table under **Median** below.
---
## Output
While running, the feed handler prints each tick and the window's statistics to stdout
in raw `int64_t` form, along with the window occupancy and the symbol. The CSV writer
converts the values and writes one file per instrument under `CSV/`, one row per trade.
The files have no header row. The columns are:
```
time,trade_id,price,mean,median,low,high
2026-09-10T15:28:16.731415Z,1091155129,77078.78,77078.77,77078.77,77078.77,77078.78
```
- `time` is the **exchange** timestamp from the ticker message, not the local receive
time.
- `price`, `low` and `high` are traded prices exactly as received. `median` is exact
at `TICK_SIZE = 1`, and otherwise the lower edge of its bucket. `mean` is truncated
toward zero to 2 decimal places, so it may understate the true mean by up to one
cent.
- Trailing zeros are trimmed, so 78101.00 prints as `78101` and 84880.50 as `84880.5`.
Column widths are therefore ragged. This is cosmetic, and the values are exact.
- The four statistics describe the window **as of that trade**, meaning the trade has
already entered the window when they are computed.
- The first row for an instrument necessarily has mean = median = low = high = price.
The `CSV/` directory is not tracked.
---
## Design
```
connect -> subscribe -> producer: read -> parse JSON -> tick -> SPSC enqueue
consumer: dequeue -> window update -> CSV
```
### Threading
Two threads joined by a lock-free single-producer single-consumer queue
(`boost::lockfree::spsc_queue`, 4096 slots). The producer owns the socket and the JSON
parser. The consumer owns the per-instrument windows and CSV writers. The split keeps
network ingest apart from computation and output, so a slow write never delays the
next read off the socket. The statistics update is a handful of integer operations,
so it stays inline on the consumer: a second queue would cost more than it saves.
If the queue is full, the producer logs an error and drops that tick. The consumer
busy-spins on the queue.
### Prices
Parsed from the JSON string straight into `int64_t` at 10^-6 scale, using
`std::from_chars` for the integer and fractional parts.
*Parser limits:* a plain decimal with at most six fractional digits and an optional
leading minus sign. No `+` sign and no exponent. A malformed price is logged and
parsed as 0.
### Timestamps
Exchange timestamps drive the window edges, not a local clock, so replaying the same
input gives identical output. They are converted to epoch nanoseconds with Howard
Hinnant's `days_from_civil`, and back again with `civil_from_days` for the CSV.
*Parser limits:* fields are read at fixed offsets, so the parser assumes zero-padded
fields, microsecond precision and a `Z` suffix. Numeric timezone offsets are not
handled.
### Sliding window
A ring buffer over a flat array with a power-of-two capacity, so wrapping is a mask.
Each push takes the right edge from the newest timestamp and expires ticks from the
front. A tick exactly one window length older than the newest stays in the window.
Capacity starts at 2048 and doubles up to a ceiling of 16,384. Past that, a tick is
refused rather than overwriting unread data. I chose growth over a fixed size with
dropping because a reallocation is an occasional copy that stops happening once the
buffer settles, whereas a dropped tick corrupts every statistic for the rest of the
window.
### Mean
A running sum, incremented on push and decremented on expiry, divided by the
occupancy.
The mean is the one statistic that is not a traded price. The sum is kept at full
10^-6 precision, and the division truncates to 2 decimal places so the column matches
the price columns. It can therefore read up to a cent below the true value.
With trailing zeros trimmed, a mean of 87912.00325 prints as `87912`. That is the
intended presentation, not a formatting failure: the value is correct at the
resolution shown.
### High / low
Two monotonic deques (`boost::circular_buffer`). When a tick arrives, smaller values
behind it in the max deque are dropped: they are shadowed by a larger value that also
expires later, so they can never be the maximum again. The min deque mirrors this.
### Median
An exact sliding-window median needs every price in the window held, so memory scales
with window length. That rules out long windows such as an hour.
Instead, the median uses a fixed-size count histogram over the price domain. A push
increments a bucket and an expiry decrements it, both O(1) and independent of window
length. To read the median, walk the buckets from the low end, accumulating counts,
and stop when the total passes n/2: that bucket holds the median trade. The scan is
bounded to `[low, high]` from the monotonic deques, so it costs the observed price
range rather than the whole array.
The bucket width is `TICK_SIZE` cents, and that alone sets the resolution: 1 keeps
both cent digits, 10 keeps tenths, and 100 discards cents.
| TICK_SIZE | bucket width | median error |
|---|---|---|
| 1 | $0.01 | **exact**, since the bucket lattice matches the exchange's cent lattice |
| 10 | $0.10 | < $0.10 |
| 100 | $1.00 | < $1.00 |
Coarser settings report the bucket's lower edge, so the error is one-sided: the true
median lies in `[reported, reported + width)`.
For an even count, the upper of the two middle values is reported rather than their
average, so the median is always a price that actually traded.
**Limitation: no rebase, and no bounds check.** The histogram is anchored to the first
price seen and never moves. A price more than `MASK` cents from that anchor falls
outside the buffer, and because the index is not bounds-checked, it writes out of
range. The fix is to recentre and rebuild when a price escapes, which is O(window) and
rare, since the deques already hold the new bounds. Not implemented.
---
## Testing
```bash
./run-test.sh
```
Tests and test data live under `test/`, using Catch2. The run prints results to the
terminal, not to a file. A Gitea Actions workflow builds everything and runs the tests
on every push and pull request.
Two generated feeds exercise the sliding window itself: a short one of 10,000 ticks
over a 1-minute window, and a long one of 1,000,000 ticks over a 1-hour window. They
check occupancy, head and tail, and which ticks remain after expiry, tick by tick.
They do not assert on the statistics.
The statistics are covered by two 20-tick fixtures with expected values worked out
independently of the code. One runs at a 5-minute window and covers the accumulate
path. The other runs at 1 second and covers expiry: multi-tick expiry, high and low
ageing out, duplicate timestamps, and a tick sitting exactly on the window boundary.
The parsers have their own vectors, 1,000,000 of each: timestamps across several
centuries with a round trip both ways, and prices with 0 to 6 fractional digits,
including sub-$1 values and whole numbers.
All vectors are generated in Python (`test/genTestData.py`) using `datetime` and
`timedelta` integer arithmetic.
---
## Known gaps
- The median histogram never rebases, and an out-of-range price indexes outside it
(see **Median** above).
- There is no signal handling. The producer blocks in the socket read, so the process
is stopped with Ctrl-C, which skips the CSV writers' final flush and loses up to
2 KB of buffered rows per instrument.
- A malformed price parses as 0 rather than being rejected.
- A full queue drops the tick with a log line rather than applying back-pressure.
## What I'd do next
- Rebase the median histogram when a price escapes the buffer, and bounds-check the
index.
- Clean shutdown on SIGINT, so the CSV buffers are flushed.
- Port to Clang and macOS.
- A second SPSC handoff for the CSV writer, if the write ever became synchronous.
---
## Third-party libraries
| Library | Use |
|---|---|
| Boost.Beast / Asio | Transport primitives for the WebSocket client |
| Boost.Lockfree | SPSC queue between the producer and consumer |
| Boost.Circular Buffer | Monotonic deques for low and high |
| simdjson | Ticker message parsing |
| Catch2 v3 | Test framework |
The WebSocket client itself (`dep/websockets`) is mine, written on top of Beast. It
handles the connection, TLS handshake, subscription and blocking receive.
Also written by hand: the sliding-window ring, the median histogram, the timestamp
conversion (a port of Hinnant's `days_from_civil` and `civil_from_days`) and the
buffered CSV writer. Integer and price conversions use `std::from_chars` and
`std::to_chars`.