Prayog technical overview

How each feature is built, how an order and its consequences flow through the system, and what a production exchange does differently. Read with the functional overview.

How to read this

• Tags match the functional page: MVP PHASE 2 LATER.
• Hot path means code that runs for every order on the matching thread. It has the strictest rules: no I/O, no locks, no system clock, as little allocation as possible.
• Build vs. library boxes. We write our own code to learn. Each component has a box showing what using an open-source library would look like, its licence, and why we did or did not use it. Licences were checked on GitHub on 3 Oct 2026.
• Exact decisions live in docs/BUILD_PLAN.md section 16 and docs/adr/. This page explains them; it does not override them.

1. Architecture

2. Principles, and why each exists

3. The order path, step by step

A bot sends BUY 10 ABC @ ₹100.05 and it partly trades. Steps 4–7 are the hot path.

1. Client → Traefik → gateway. HTTPS POST /orders with a JSON body and a bearer token. Traefik routes by hostname. MVP runs plain HTTP locally.
2. Gateway checks (on a WebFlux event-loop thread): verify the JWT signature with Keycloak's public key (cached; no network call per request), take a token from the account's rate-limit bucket, check the body against new-order-request.schema.json rules, and turn JSON into a NewOrder command. A failure here returns HTTP 4xx with an RFC 9457 problem body. The engine never sees it.
3. Publish to the ring. The gateway claims the next slot in the Disruptor ring buffer, copies the command in and publishes it. This is a few atomic operations with no lock. If the ring is full the gateway rejects with 503 ("back-pressure") instead of queueing without limit.
4. Sequence. The matching thread takes the command and gives it the next input sequence number. That number fixes its place in history.
5. Validate. Symbol, quantity, tick, band, session state, account enabled. The first failure gives OrderRejected(reason).
6. Match. Look at the best ask level. While it crosses the limit, fill against the oldest order there, emit a Trade, and remove filled orders and empty levels. Then rest the remainder at the back of its price queue.
7. Emit events to the event sink, each with the next event sequence number and the sim time: OrderAccepted, then the Trades.
8. Make durable. The journal handler appends the input command and its events to files. At the end of each Disruptor batch it calls fsync once, so many orders share one disk flush ("group commit").
9. Respond and publish, only after step 8: the gateway completes the HTTP response, the market-data handler pushes book changes and trade prints to WebSocket subscribers, and the owner gets private order updates.
10. Kafka publisher (own thread) reads new records from the event log and sends them to Kafka, keyed by symbol. If Kafka is down it simply falls behind and catches up later; trading never waits for it.
11. Post-trade (another service) consumes the trades, updates positions, P&L and charges in PostgreSQL inside one transaction per event batch, and updates the leaderboard in Redis.

4. Feature → mechanism map

5. Components

5.1 Gateway MVP S10

What it does: the only door into the exchange. It authenticates, rate-limits, validates shape, converts JSON to commands, and sends results and feeds back out. It holds WebSocket sessions and nothing about books.

Why WebFlux: it handles thousands of connections on a few event-loop threads. That suits thousands of long-lived WebSockets better than one thread per connection. Spring's own WebSocket support means no extra server.

Key design points

• Validate the JWT locally (cached public keys from Keycloak). Calling Keycloak per request would put the network on every order.
• Token-bucket rate limit, written by hand (BUILD_PLAN 16.1 #22). It is ~30 lines, and writing it teaches the algorithm.
• Never block an event-loop thread. Publishing to the ring is non-blocking. If the ring is full, fail fast with 503.
• Correlate responses by clientOrderId. The engine's reply comes back on another thread, and the gateway completes the waiting HTTP response.

Bucket bucket = Bucket.builder()
    .addLimit(limit -> limit.capacity(50).refillGreedy(50, Duration.ofSeconds(1)))
    .build();
if (!bucket.tryConsume(1)) {
    return problem(429, RejectReason.RATE_LIMITED);
}

public void fromApp(Message msg, SessionID session) {
    if (msg instanceof NewOrderSingle nos) {
        ring.publish(toNewOrder(nos, accountFor(session)));   // same command as REST
    }
}

5.2 Sequencer pipeline (Disruptor) MVP S7

What it does: turns many concurrent producers (gateway threads, the clock thread, ops) into one ordered stream for the matching thread.

New to ring buffers? The ring buffer, explained walks through a business example, the mechanics, an interactive simulator and Prayog's code.

Key design points

• Pre-allocated ring of 65,536 mutable slots. Producers copy data in; nobody allocates per message, so the garbage collector has little to do.
• Handlers form a dependency graph. Journal and publishers run after matching on the same slots, in parallel with each other, each on its own thread.
• Wait strategy: BLOCKING (low CPU, for laptops) by default; BUSY_SPIN burns a whole core to shave microseconds (benchmarks).
• Batching for free: a handler that falls behind sees many ready slots and processes them as a batch. That is where group-commit fsync comes from.

5.3 Matching engine and order book MVP S4–S6

What it does: applies the exchange rules to each command, keeps the books and emits events. Pure Java, no framework, no I/O: it can be tested with plain unit and property tests.

Data structures (built in S4)

OrderBook (one per symbol)
  bids : TreeMap<price, PriceLevel>  highest first
  asks : TreeMap<price, PriceLevel>  lowest first
  ordersById : HashMap<orderId, RestingOrder>

PriceLevel                         RestingOrder (is the list node)
  head ⇄ order ⇄ order ⇄ tail        orderId, accountId, side, price, leaves
  totalQuantity, orderCount          prev, next, level

*In practice L is small (hundreds), so the tree is shallow. Benchmarks in S9 decide whether to change it.

What production engines do differently (and when we would follow)

How it is tested

• Unit tests: one per rule, readable as a specification.
• Property tests (jqwik): random order flows, checking that the book is never crossed, quantity is conserved, sequences have no gaps, and runs are deterministic.
• Reference model: a naive list-scanning matcher must produce identical events. It catches "plausible but wrong" bugs like LIFO instead of FIFO.
• checkInvariants() walks the whole structure after each step in tests.

ExchangeCore core = ExchangeCore.builder()
    .resultsConsumer(new SimpleEventsProcessor(handler))   // trade / reduce / reject callbacks
    .exchangeConfiguration(ExchangeConfiguration.defaultBuilder().build())
    .build();
core.startup();
core.getApi().submitCommandAsync(ApiPlaceOrder.builder()
    .uid(301L).orderId(5001L).price(10_005L).size(10L)
    .action(OrderAction.BID).orderType(OrderType.GTC).symbol(ABC_ID)
    .build());

5.4 Simulated clock MVP S7

A small thread off the order path reads wall time every 100 ms and publishes a ClockTick(simTime) command into the ring, where simTime = sessionStart + wallElapsed × multiplier. The engine keeps "current sim time" = the last tick. Session changes (09:15 open, 15:30 close) happen when a tick crosses the boundary.

Why ticks are commands: they are journaled like orders, so a replay sees the exact same times in the same places, even if the replay runs 1,000× faster than real time. The wall clock is read in exactly one place, and that place is outside the engine.

Resolution trade-off: every event between two ticks shares a sim time; the sequence number gives the order. Finer ticks mean more journal records. 100 ms is plenty for a human-speed market.

5.5 Journal and replay MVP S8

Two append-only logs (BUILD_PLAN 16.1 #14):

• Input journal: every command in sequence order. Replaying it rebuilds the engine state exactly. This is event sourcing: the log is the truth, and memory is a cache of it.
• Event log: every event produced. It is used for the determinism checksum and as the source the Kafka publisher reads.

record = [length:int][seq:long][type:short][payload:bytes][crc32c:int]
segments: /data/journal/input-000001.log, input-000002.log, …   (roll at a size limit)

• CRC32C detects a torn final record after a crash: replay stops at the first bad checksum.
• fsync per batch: the OS caches writes in memory, and only fsync guarantees they reached the disk. One fsync per Disruptor batch amortises the ~1 ms cost.
• Replay check in CI: run a recorded session, replay it, and compare SHA-256 of the two event logs.
• LATER Snapshots: replaying a whole day is slow, so periodically serialise the books with the sequence number they reflect, then replay only from there.

try (ChronicleQueue queue = ChronicleQueue.singleBuilder("/data/journal/input").build()) {
    ExcerptAppender appender = queue.createAppender();
    appender.writeDocument(w -> w.write("newOrder").marshallable(cmd));
    // replay:
    ExcerptTailer tailer = queue.createTailer();
    while (tailer.readDocument(r -> engine.apply(readCommand(r)))) { }
}

5.6 Market data MVP S11

• Snapshot + deltas: on subscribe, send book.snapshot (all levels, with the seq it reflects), then book.update for each level change. The client applies deltas with seq > snapshot seq.
• Gap detection: per-symbol seq numbers; a gap means "re-request the snapshot". This is the same pattern as exchange multicast feeds with a recovery channel.
• Slow consumers: each WebSocket has a bounded outgoing buffer. If it fills, the client is too slow, so drop its queued updates and send a fresh snapshot (conflation), or disconnect it. Never let one slow browser hold memory for ever or slow the publisher.
• Privacy: public trades carry no account IDs (the schema forbids them); private order updates go only to the owner's sessions.
• The engine emits BookUpdate from the level totals it already keeps, so market data never reads the book from another thread (which would need locks).

5.7 Kafka publisher MVP S15

Kafka in one paragraph: a topic is an append-only log split into partitions. Producers append; each record gets an offset (its position in the partition). Consumers read in order and periodically save ("commit") the offset they reached, so after a restart they continue from there. Order is guaranteed only within one partition, so we key by symbol: all ABC events land in one partition, in order.

• Source: the publisher tails the event log, not the ring, so it can be minutes behind without slowing anything. It remembers "last published seq" and resumes from seq + 1 after a restart or Kafka outage.
• At-least-once: if the publisher crashes after Kafka stored a record but before it noted the seq, it sends that record again. Duplicates are possible; loss is not.
• Producer settings: acks=all (written to all replicas before success), idempotent producer on (Kafka drops its own retry duplicates within a session).
• Consumers must be idempotent: because of at-least-once delivery, post-trade skips events whose seq it has already applied (section 5.8).

5.8 Post-trade service MVP S16

• Consumes Trade events; for each trade updates two accounts (buyer and seller).
• Exactly-once effect from at-least-once delivery: in one database transaction, insert the event's seq into applied_event (primary key) and update positions. A duplicate fails the insert, so it is skipped. The Kafka offset is committed after the transaction.
• Positions: (account, symbol) → quantity, total cost in paise. Average price = cost ÷ quantity, computed when shown, so nothing is rounded in storage.
• Unrealised P&L needs the last price: kept from trade prints in memory or Redis, not recomputed in SQL per tick.
• jOOQ + Flyway: SQL-first, type-safe queries generated from the migrated schema; migrations run once as a job before services start.

5.9 Leaderboard MVP S17

A Redis sorted set keeps members ordered by score: ZADD prayog:leaderboard 931.50 acct-7. "Top 20" is ZREVRANGE … 0 19 WITHSCORES, O(log N + 20). Redis is a cache here: it can be rebuilt from PostgreSQL at any time.

5.10 Simulated traders and SDK MVP S12–S13

• SDK: async Python client. It gets a token, places orders over REST, streams over WebSocket, keeps a local book from snapshot + deltas, and checks seq gaps.
• Agents: one asyncio process running many traders. Each trader has a strategy and a seeded random generator. A hidden "fair value" per symbol follows a random walk with occasional jumps, and market makers quote around it.
• They connect like any user (client credentials), which load-tests the gateway for free.

5.11 Web terminal MVP S18–S19

• React + Redux Toolkit. RTK Query handles REST (orders, blotter, leaderboard); a WebSocket middleware feeds book and trade updates into the store.
• Book rendering is throttled to animation frames: updates may arrive faster than 60 per second, and nobody can read faster.
• All prices are integers in the store, formatted as ₹ only in the view. Every integer from the server is below 2⁵³ by schema, so JavaScript numbers are exact.
• Charts: TradingView Lightweight Charts (attribution link required).

5.12 Observability MVP S21

• Micrometer metrics → Prometheus → Grafana: orders/s, rejects by reason, ring fill level, journal fsync time, Kafka lag (last journaled seq − last published seq), consumer lag.
• Latency as HdrHistogram percentiles (p50/p99/p99.9/max), never averages: averages hide the tail that traders feel.
• Structured JSON logs to stdout, with the order's seq so one order can be traced across services.

6. Failure modes

7. From MVP to production

The MVP is architecturally faithful to production exchanges. Here is what they add:

public final class ExchangeService implements ClusteredService {
    private final MatchingEngine engine = new MatchingEngine(instruments, this::reply);
    @Override
    public void onSessionMessage(ClientSession session, long timestamp,
                                 DirectBuffer buffer, int offset, int length, Header header) {
        engine.submit(decode(buffer, offset, length), simTime);   // same deterministic engine
    }
    // Aeron Cluster replicates each message to every node (Raft) before delivering it,
    // so all nodes run the same inputs in the same order and hold the same books.
}

8. Libraries: use or learn from

Adding any library not already in BUILD_PLAN section 8 still needs your approval first (CLAUDE.md).

9. Open technical decisions

Sources

• Martin Fowler, "The LMAX Architecture" (martinfowler.com/articles/lmax.html): single-threaded business logic, event sourcing, Disruptor.
• LMAX Disruptor technical paper and repository: github.com/LMAX-Exchange/disruptor (Apache-2.0).
• exchange-core: github.com/exchange-core/exchange-core (Apache-2.0): README, OrderBookNaiveImpl, OrderBookDirectImpl.
• CoralME: github.com/coralblocks/CoralME (Apache-2.0).
• Liquibook: github.com/enewhuis/liquibook (OCI licence).
• WK Selph, "How to Build a Fast Limit Order Book" (2011): tree of levels + linked lists + order map.
• Aeron and Aeron Cluster: github.com/aeron-io/aeron (Apache-2.0); Agrona: github.com/aeron-io/agrona.
• Chronicle Queue: github.com/OpenHFT/Chronicle-Queue (Apache-2.0).
• Apache Kafka documentation: delivery semantics, idempotent producer.