Prayog: scale to 1 million users¶

Prayog today is one JVM with one shard, and that is the right size for a simulated market at tens of commands a second. This note shows how the same design grows to 1 million registered users without changing the matching rules. It is a design, not a build plan: every capacity number is either measured (with a link to docs/benchmarks.md) or an estimate marked as one.

As of 2026-10-08.

The answer in one paragraph¶

Shard by market: each shard owns a group of symbols and is still one single-writer engine with its own journal, so nothing inside the order path changes. Make each shard highly available with a hot standby that replays the same input journal (determinism makes the standby's state byte-identical). Then scale the two things that really grow with users, which are the gateways and market data fan-out. Gateways are stateless. Market data goes through a fan-out tier that conflates updates for browsers and applies backpressure per subscriber, so a slow client never slows the exchange.

Assumptions (estimates)¶

Quantity	Value	Why
Registered users	1,000,000	The target
Daily active	100,000 (10%)	Typical for a free simulation game
Peak concurrent connections	50,000 (5%)	Market open and the leaderboard reset
Peak human orders	2,500/s	50,000 × 1 order every 20 s
Peak bot orders	10,000/s	5,000 user bots × 2/s (the `bot` tier allows 50/s; most use far less)
Simulated traders	500/s per shard	The `agent` tier (ADR 0020)
Design peak	50,000 commands/s	About 4x the sum above, for bursts and growth
Symbols	200	NSE-like large caps plus a few indices

What one shard can do today (measured)¶

Measure	Number	Source
Matching cost	about 60-120 ns per command, one thread	`docs/benchmarks.md`, S9 JMH
Pipeline throughput with journal and fsync	20,000 commands/s held, p50 12.7 ms	S9 HdrHistogram, Mac SSD
fsync in the Docker volume	0.8 ms per 4 KiB `O_DSYNC` write	2026-10-08 entry
Journal record size	about 51 B per input, 67 B per event	64 MiB segments: 1,319,912 inputs, 1,002,939 events
Archive ratio	2.6x (input), 4.8x (events)	ADR 0022 entry

So one engine thread could match millions of commands a second. The limits are the durable write (fsync or replication) and everything after the engine: answering clients and publishing market data.

The shape at 1 million users¶

flowchart LR
  subgraph Edge
    LB[Load balancer] --> GW1[Gateway 1..n<br/>stateless, JWT offline]
  end
  GW1 -- route by symbol --> S1
  GW1 -- route by symbol --> S2
  subgraph S1[Shard A: symbols 1-50]
    P1[Primary: sequencer + engine] -- input records --> B1[Hot standby: replays]
    P1 --> J1[(Journal)]
  end
  subgraph S2[Shard B..N]
    P2[Primary] -- input records --> B2[Hot standby]
  end
  P1 -- events --> MD[Market-data fan-out tier]
  P2 -- events --> MD
  MD --> WS[WebSocket edges<br/>conflated for browsers<br/>full feed for bots]
  P1 -- events --> K[(Kafka)]
  P2 -- events --> K
  K --> PT[Post-trade, partitioned by account] --> PG[(Postgres, sharded by account)]
  PT --> R[(Redis leaderboard)]

1. Shard by market¶

Unit of sharding: a group of symbols. An order touches exactly one symbol, so matching never crosses shards and each shard keeps the rule "one writer thread, no locks" (CLAUDE.md engine invariants).
How many: at 50,000 commands/s and 20,000/s per shard measured on a laptop SSD, 4 shards with headroom; 8 if the busiest symbols are uneven. Hot symbols get a shard of their own (Reliance on budget day).
Routing: the gateway holds a symbol-to-shard map. Moving a symbol between shards happens at session close: the old shard snapshots the book, the new shard loads it (the snapshot format exists, ADR 0016).
Sequencing: each shard has its own input sequence and event sequence. There is no global order across shards, and none is needed: a trade involves one book.
What crosses shards: per-account controls. The kill switch (SetAccountEnabled) is broadcast to every shard as a command, so each shard journals it in its own order. Cross-symbol risk limits (total exposure) need either a credit split per shard (each shard gets a share of the account's limit, re-balanced between sessions) or a pre-trade risk gate in front of the shards. The credit split keeps the order path free of network calls, which is an invariant.

2. Hot standby that replays the journal¶

Why it works: the engine is deterministic. The same inputs in the same order produce a byte-identical event log (make replay-check and make e2e prove this today). So a standby that applies the primary's input records is an exact copy, without copying any engine state.
Replication instead of fsync: today a client is answered after fsync (12 ms on the Mac, 0.8 ms in the container). In the cluster the primary sends each batch of input records to the standby and answers once the standby has them in memory; both still write to disk asynchronously. Two machines failing in the same millisecond is less likely than one disk losing an unflushed write. This is what LMAX and Aeron Cluster do, and it moves the latency floor from the disk to the network (tens of microseconds in one data centre).
Failover: one leader per shard, chosen by a lease (for example etcd or a Raft group of three). The standby is promoted only after the lease expires, so two primaries never run at once. Clients reconnect and re-send with their client order IDs; duplicates are rejected per account per day (ADR 0014), so a retry never trades twice.
Catch-up: a new standby starts from the newest verified snapshot (ADR 0016) and replays the journal after it, including archived segments if needed (ADR 0022).

3. Market data fan-out with conflation and backpressure¶

This is where 1 million users really cost something.

The raw number (estimate): 50,000 commands/s over 200 symbols is about 250 book changes a second per symbol. A browser watching 5 symbols would get about 1,250 messages a second, about 190 KB/s of JSON. For 50,000 connections that is about 9.4 GB/s. Not possible, and no person can read it anyway.
Conflation for browsers: the fan-out tier keeps the latest state per symbol and sends at most 10 updates a second per symbol. Book changes between sends are merged (the latest quantity per price level wins); trades are summarised as last price, volume and count. 5 symbols × 10/s × about 150 B is 7.5 KB/s per browser, about 375 MB/s (3 Gbit/s) in total, spread over about 20 edge nodes.
Full feed for bots: every delta with its seq, as today. Limited to bots that ask for it, and to the symbols they trade.
Backpressure, two kinds:
Conflated subscribers have one slot per symbol, not a queue. A slow client simply skips intermediate states and always gets the newest. It never falls behind, so it is never disconnected for being slow.
Full-feed subscribers have a bounded queue. A subscriber too far behind is disconnected and must take a fresh snapshot (today: more than 10,000 messages behind, ADR 0009). The exchange never waits for a client.
Removing the lock (the lock-free change): today subscription and publication share one lock in MarketHub, so a new subscriber's snapshot and its first delta are consecutive (ADR 0009 #5). At scale the fan-out tier uses the standard sequenced-feed approach instead: the client subscribes to deltas first, buffers them, fetches a snapshot that carries its seq, then drops buffered deltas with seq at or below the snapshot's. No lock is shared between the publisher and snapshot readers, and a snapshot can be served from a cache.
Transport between shards and the fan-out tier: a broadcast log per shard (Aeron or UDP multicast in a data centre; Kafka with short linger is simpler, at a few milliseconds more). The fan-out tier is stateless apart from the current books, which it rebuilds from a snapshot plus deltas on restart.

4. Everything after the trade¶

Kafka: partitions keyed by account, so one account's events stay in order. Kafka stays off the order path, as today (the publisher tails the durable event log, ADR 0013).
Post-trade ledger: Postgres sharded by account id (the id is already a 63-bit hash, so ranges spread evenly). The consumer is idempotent, so redelivery after a failover changes nothing (ADR 0015).
Leaderboard: one Redis sorted set holds 1 million members easily (ZADD and rank are O(log n)). The top 100 is cached for a second; "my rank" is one ZREVRANK.

5. Gateways and limits¶

Stateless: JWT checked offline against cached JWKS keys, so login traffic hits Keycloak, not every order.
Rate limits per tier (ADR 0020) need shared state across gateways. Route each account to one gateway (consistent hashing on account id) so its token bucket stays local; a gateway failure resets a few buckets, which is harmless.

Stages¶

Stage	Users	Change	Main risk
1 (today)	100s	One JVM, one shard, fsync per batch	Host memory (see the GC entry in `docs/benchmarks.md`)
2	10,000s	Gateway and market data in their own processes; hot standby; conflated browser feed	Failover tested only in drills
3	1,000,000	4-8 shards by symbol group; replicated journal; fan-out tier of about 20 edges; post-trade sharded by account	Cross-shard risk limits; rebalancing hot symbols

Trade-offs to defend¶

Shard by market, not by account: a book must live in one place to match, so the shard key is the symbol. The price is cross-symbol risk, handled by a credit split.
Replication over fsync: lower latency and still durable against one failure, but the system now depends on the network and a correct leader election. The lease and client order IDs keep this safe.
Conflation: browsers see fewer updates. People cannot read 1,000 updates a second; bots that need every update use the full feed.
Deterministic replay as the backbone: the standby, recovery, the replay check and audit all use it. Its cost is discipline: no system clock, no floating point, no iteration over unordered collections in the engine.

Open questions¶

Which consensus option for the lease: etcd (simple) or a Raft group inside the shard (no extra system)?
Is Kafka fast enough between shards and the fan-out tier, or is Aeron worth its operational cost?
How often to rebalance the credit split for cross-symbol limits?