Skip to content

Choosing Messaging for a Matching Engine: Aeron® vs Kafka vs Chronicle Queue

If the matching engine must survive a node loss without losing a committed order, Aeron® Cluster is the only option on this list that ships a Raft-replicated state machine for your own code. You get consensus, a replayable log and UDP transport in one Apache-2.0 stack, and you pay for it with deterministic-code discipline and JVM low-latency ops. Apache Kafka is built for the other half of an exchange: retained, re-readable event streams over TCP for risk, analytics, surveillance and audit, with commit latency its own docs put “on the order of 10 ms”. Chronicle Queue and the LMAX Disruptor are single-host building blocks. Neither replicates your state across machines unless you build that yourself, or buy Chronicle’s Enterprise replication.

Comparison: the properties that decide an exchange design

Section titled “Comparison: the properties that decide an exchange design”
propertyAeron (Transport + Archive + Cluster)Apache KafkaChronicle Queue (OSS)LMAX Disruptor
Replicated state machine for your codeYes. Cluster runs your service as a replicated state machine on RaftNo. It replicates the log; your consumer’s state is yours to rebuildNo. The README describes Enterprise replication for “real-time backup”, not consensusNo
Commit / consensus modelRaft majority commitLeader + dynamic in-sync replica set (ISR), not majority vote; KRaft quorum for metadata onlyNone (single writer, local files)None
Persistence & replayArchive records streams for “later, or real-time, replay”; default sync level is page cache onlyDurable log, per-topic time retention, consumers rewind to any retained offsetMemory-mapped files on local disk; no network file systemsNone (in-memory ring buffer)
TransportUDP unicast, UDP multicast, IPCTCP binary protocolSame-host shared files; cross-host only via Enterprise replicationInter-thread, in one process
Typical deployment3-node cluster + clients, pinned cores, busy-spinBroker cluster (RF 3 common) + 3 or 5 KRaft controllersLibrary inside your JVMsLibrary inside one JVM
LanguagesJava, C, C++ clients (.NET separate); Cluster service is JavaJVM: Java client, brokers in Java and some ScalaJavaJava
Licence & supportApache-2.0; Adaptive sells Premium components, training, supportApache-2.0Apache-2.0; Enterprise Edition is commercialApache-2.0
What it is NOT forLong-retention fan-out to dozens of consumer teamsA µs-latency, synchronously replicated order bookMulti-host HA out of the boxAnything that leaves the process

Treat the numbers in the next sections as not comparable: each comes from a different vendor, rig, message size and harness. None of them was measured head-to-head on the same hardware.

Pick Aeron Cluster when sequencing and state replication are the product, which is true of a matching engine. The README describes Cluster as “fault-tolerant services as replicated state machines based on the Raft consensus algorithm”. In practice: every node applies the same ordered log, so a follower holds the same order book as the leader at the same log position. See Aeron Cluster and Raft Consensus for the five safety guarantees that make that true, and The Aeron Files for the election internals.

What the numbers on this site look like (our rigs, not vendor claims):

  • p50 37 µs, 1 publisher → 3 MDC receivers, 224 B at 50k msg/s, bare metal. Tuning the NAK timers took p99.9 under 0.01% seeded loss from 8,286 µs to 105 µs (NAK Timer Tuning).
  • Leader failover ~11 s stock, ~1.5 s with a 1 s aeron.cluster.leader.heartbeat.timeout, 200 ms heartbeat interval and 500 ms election timeout. Aeron 1.48.11, bare metal (Tuning Cluster Failover Time).
  • Benchmark latencies depend heavily on the burst knob. Read Reading the Aeron Benchmark Honestly before you put a vendor or community number in a slide.

The costs, which decide more stacks than latency does:

  • Deterministic service code is mandatory. Wall-clock time, randomness or HashMap iteration order in the service silently diverges replicas (Runbook: Snapshots & Determinism).
  • The clustered service is Java. In the OSS repo, aeron-cluster/src/main holds only Java sources (java/, plus resources/). Clients exist in Java, C and C++, with a separate .NET client.
  • Low-latency ops is a skill set. Isolated cores, busy-spin idle strategies and driver tuning are how you hold p99 (Core Isolation & Thread Pinning, Smart Batching and Idle Strategy).
  • Retention is something you operate. The archive API will delete cluster-log segments without checking whether a node still needs them to restart, so purging needs a procedure (Archive Retention & Purge). Durability defaults to the page cache (Archive Durability and Replay).
  • Some features are commercial. The README lists “kernel bypass with DPDK” and encryption with ATS as proprietary enhancements from Adaptive, the company behind Aeron Premium. See When You Need DPDK before you assume you need it.

Pick Apache Kafka for retained streams that many consumers re-read. The intro describes storing events “durably and reliably for as long as you want”, with retention set per topic. The design doc says “a consumer can deliberately rewind back to an old offset and re-consume data”. Clients and brokers speak “a high-performance TCP network protocol”.

Why it is not a matching-engine core:

  • Commit latency. Kafka’s design doc: waiting for a message to be committed “can take on the order of 10 ms”. You can trade durability down with acks (async, or leader-only), which is a durability decision, not a tuning knob.
  • Replication model. “Instead of majority vote, Kafka dynamically maintains a set of in-sync replicas (ISR)”. With acks=all, the ack comes when all current ISR members have the record; min.insync.replicas sets the floor. It is a sound log-replication design. It is not a replicated state machine for your matching logic: on failover, your consumer rebuilds state from offsets.
  • Ordering is per partition. Topics are “divided into a set of totally ordered partitions”. A global sequence across instruments needs a single partition or your own sequencer.
  • Batching is the throughput lever. The doc’s example bound is “64k or 10 ms”. That is the right trade for streams and the wrong one for an order-entry hot path.

Chronicle Queue is “a broker-less, off-heap Java library for ultra-low-latency, persisted messaging”, storing data in memory-mapped files. It fits same-host pipelines and journaling: gateway → engine → publisher processes on one box, with every message persisted and replayable.

  • Vendor figure, same machine: 99th percentile 0.78 µs at 10 million events/minute, 40-byte messages, “one thread to publish, and one thread to consume” (Chronicle README; hardware not stated).
  • No network file systems. The README says Chronicle Queue “does not support operating off any network file system”, and “the only supported way” to reach multiple hosts is Chronicle Queue Replication, an Enterprise Edition feature.
  • No consensus. Replication is described as “real-time backup of all your queue data”. Nothing in the README describes leader election or majority commit. Failover logic is yours to design.

Choose it over Aeron when the engine is single-host by design and HA is handled another way, or you already license Chronicle Enterprise. Choose against it when multi-node fault tolerance has to come out of the box with an Apache-2.0 licence.

The LMAX Disruptor is a different category

Section titled “The LMAX Disruptor is a different category”

The Disruptor is “a High Performance Inter-Thread Messaging Library”. It moves events between threads inside one process. It has no network transport, no persistence and no replication, so compare it with ArrayBlockingQueue, not with Kafka or Aeron. LMAX’s paper reports a mean of 52 ns per hop against 32,757 ns for ArrayBlockingQueue (three-stage pipeline, Core i7-2720QM at 2.2 GHz, Java 1.6.0_25, Ubuntu 11.04, no CPU binding).

It answers a question inside your engine, the threading model between gateway decode, matching and publishing, and it sits alongside whichever transport you choose. Durability and failover come from somewhere else.

productwhat its docs sayfit for a matching-engine core
NATS (Apache-2.0)Core NATS is “at most once, never replayed”. JetStream adds persistence and replay; each replicated stream is its own Raft group and writes are acked once a majority holds themJetStream has real consensus, but it is a broker you call over the network. Your state machine still lives in your process
ZeroMQ (libzmq, MPL-2.0)“Looks like an embeddable networking library but acts like a concurrency framework”, with sockets over in-process, inter-process, TCP and multicastTransport only. Persistence, replay and failover are yours to build
SolaceEvent broker. “Direct messaging” is the default, “reliable, but not Guaranteed”; Guaranteed messages sit in a non-volatile storeA broker hop on the hot path, with no replicated state machine for your code
  1. Must a committed order survive a node loss with no replay gap? Yes → you need consensus over your state machine: Aeron Cluster, or a Raft layer you build yourself on top of the others.
  2. What is the round-trip budget for order entry? Microseconds → UDP/IPC transport, pinned cores. Several ms is fine → Kafka or a broker is viable, and simpler to staff.
  3. How many consumer teams, and how long is retention? Many teams, weeks of history → Kafka downstream, whatever runs the engine.
  4. Is the engine single-host by design? Yes → Chronicle Queue or the Disruptor plus your own HA plan. Write that plan down before you commit.
  5. Who owns determinism reviews and JVM tuning? Nobody → don’t pick Aeron Cluster yet. Its failure modes are silent divergence and tail spikes, and both need a named owner.
  6. Will you scale past one sequencer? Plan the shard key now (Sharding by Actor Pattern).
  7. Measure on your own rig before you sign anything. Every number above came from a different harness.

This site is not affiliated with, endorsed by, or sponsored by Adaptive Financial Consulting Limited or the Aeron project. Aeron is a registered trademark of Adaptive Financial Consulting Limited.

Aeron is a trademark of Adaptive Financial Consulting Limited in the United Kingdom and other countries.