Throughput & Latency Model

A predictive model for peak TPS, the MTU relationship, and why peak TPS is not latency, derived from the Aeron driver’s send path and validated against our benchmark runs. Use it to answer one question: “will config X saturate at load L, and what latency will I get?”

Provenance & confidence

The structure below — one MTU per syscall, one syscall per follower, a single sender thread — is proven from the driver source (NetworkPublication.sendData → SendChannelEndpoint.send). The constant S ≈ 400K sendmsg/s is a measured operating point on Intel m7i (≈2.5µs/sendmsg), corroborated by our m7i-metal runs (~725K msg/s back-solves to ~380–400K syscalls/s) — not a directly-benchmarked hard peak. Treat absolute numbers as ±~15% and the knee as order-of-magnitude; the shapes and rankings are solid and match every run. This page is the operational model — for the driver internals, see The Aeron Files.

Notation — S, μ, λ, ρ (and why ρ is a ratio, not a rate)

symbol	name	units	what it is
S	peak sendmsg/s per thread	rate (/s)	hardware/send-path constant, ≈ 400,000/s on Intel m7i. CPU-family-specific (see below) — measure per platform.
C	consensus-control syscalls/s	rate (/s)	budget stolen by control packets (~160K noop, ~75K sleep-ns).
D	followers	count	destinations per log datagram (each = D syscalls). 2 for a 3-node cluster.
M_pkt	messages per datagram	count	floor(usable_MTU / framed_msg). The MTU multiplier.
μ (mu)	capacity = peak_TPS	rate (msg/s)	(S−C)/D × M_pkt. “How fast it can go.” ≈ 790K at 128B/1408.
λ (lambda)	offered load	rate (msg/s)	what you’re actually pushing. e.g. 800K.
ρ (rho)	utilization	dimensionless	λ / μ — the operating point.

ρ is NOT the peak sendmsg rate. It’s a pure number = offered load ÷ capacity: ρ = 0.5 → 50% of capacity; ρ = 1.0 → load equals capacity; ρ > 1 → overloaded. S is a rate (an input to μ); ρ is a ratio (λ/μ). Same box and message, different load → different ρ (128B/1408: 500K→ρ≈0.63, 800K→ρ≈1.01 — same μ, only λ moved).

You lower ρ two ways: raise μ (bigger MTU → more M_pkt, or DPDK → higher S) or lower λ (send less). Mnemonic: S and μ are the speedometer’s max; λ is your current speed; ρ is the fraction of redline you’re at.

S is CPU-family-specific — the peak table below is Intel-m7i-only

S is the single-thread sendmsg syscall rate, set by the per-core cost of the kernel send path (syscall entry/exit + UDP/IP stack + NIC-driver enqueue). That cost depends on CPU microarchitecture (IPC, clock, cache/memory latency) and the platform NIC path — so S differs across vendors and must be measured per target, not assumed:

platform	typical CPU	S (single-thread sendmsg, stock path)
Intel (m7i — our cluster benchmark)	Sapphire Rapids 8488C	~400K/s (measured basis, stock NIO sendmsg)
AMD (m7a / m8a)	EPYC Genoa / Turin	likely higher (strong IPC + boost) — measure it
Graviton (m7g / c8g)	ARM Neoverse V1/V2	different — depends on generation / clock

Direction isn’t universal (it depends on clock, x86-vs-ARM kernel path, ENA generation). What IS universal is the formula and its shape — only the constant S shifts, which moves every peak number and thus where the knee sits:

• Lower S (e.g. some Graviton) → lower peak_TPS → saturates at a lower load, the knee arrives sooner. A config safe at ρ≈0.6 on m7i could be at ρ≈1 on a lower-S platform.
• Higher S (e.g. EPYC) → more headroom at the same MTU.

Consequences: (1) the worked table below is Intel-m7i-specific — re-derive S for other silicon; (2) the MTU fix transfers everywhere (M_pkt is CPU-independent), but how badly you need it is platform-dependent; (3) always measure S on your own target before quoting a peak.

The bottleneck: one thread, one syscall per MTU per follower

The leader replicates the Raft log with a single driver Sender thread. From the source:

• sendData(): scanLimit = min(availableWindow, mtuLength) → transmits at most one MTU per doSend(), and doSend = one datagramChannel.write() = one sendmsg syscall.
• The log channel is MDC: SendChannelEndpoint.send does one write per follower destination — so each log datagram costs D syscalls (D = number of followers).
• One thread tops out at S ≈ 400K sendmsg/s (the syscall path, ~2.5µs each). More cores or a bigger box do not raise this — it’s one thread.

Two single-threaded stages feed the wire, each pinned to its own core: ConsensusModule.appendMessage (writes each message once into the single log term buffer) → Sender (transmits that one buffer). The Sender is the slower link.

Peak TPS formula

                 (S − C) × M_pkt
   peak_TPS  =  ─────────────────
                        D

term	meaning	typical
S	single-thread sendmsg ceiling	~400,000 /s
C	consensus-control syscalls stolen from the budget	~150–175K (noop idle); ~75K (sleep-ns)
D	followers (each datagram = D syscalls)	2 (3-node cluster)
M_pkt	messages per datagram	floor(usable_MTU / framed_msg)

Framing (matters for small messages):

framed_msg = align32(msg_size + 32)      # 128B → align32(160) = 160B
usable_MTU = MTU − ~40 (UDP/Aeron header)
M_pkt      = floor(usable_MTU / framed_msg)

Key inversion: more followers divide the ceiling

More followers (D) divides the ceiling — the single sender serializes the per-follower writes, so 2 followers halve throughput, they don’t double it. More cores don’t help; only fewer syscalls (bigger M_pkt) or a faster send path (DPDK) do.

Worked table (S=400K, C=160K → log budget 240K/s, D=2 → 120K datagrams/s, Intel m7i)

msg	framed	MTU 1408 → peak TPS	MTU 8192 → peak TPS
32B	64B	21 msg/pkt → ~2.5M	127 → ~15M
128B	160B	8 → ~960K	50 → ~6.0M
224B	256B	5 → ~600K	31 → ~3.8M
512B	544B	2 → ~240K	15 → ~1.8M
1344B	1376B	~1 → ~120K	5 → ~600K

Retrodicts every run: 128B/1408 ≈ 960K peak vs measured ~790K ceiling / 725K achieved (ρ≈1, saturated) ✅; 128B/8192 ≈ 6M → 800K is ρ≈0.13 (never saturates) ✅; 224B/1408 ≈ 600K < 800K (must saturate) ✅.

6M at 128B/8192 is the sender-count ceiling, NOT achievable throughput

Once MTU removes the sendmsg bottleneck, a different limit binds first — the ingress flow-control window (rcv.initial.window.length), byte-bandwidth / NIC, or the consensus append thread — probably in the ~1–2M range (unmeasured). The 6M figure only proves that 800K has huge margin.

Same formula, every role — the leader just loses on D, C and frame size

Every Aeron endpoint has a driver Sender thread, so this is really the single-thread sender capacity formula, and it applies to all three roles — you just plug in role-specific values. Three terms change, and one role barely uses it:

role	its sender sends	D	C	framed unit → M_pkt	binding?
Client	ingress msgs → leader only	1	0 (not a Raft participant)	raw app msg (128B)	rarely — ~2× the leader’s capacity → ρ≈0.5
Leader	Raft log → all followers (MDC)	= followers (2)	~160K (commit/append pkts)	log frame (app msg + cluster hdr, larger)	YES — the bottleneck
Follower	SM / ack → leader only	1	small	tiny control frames (~KB/s)	never — its limit is its receiver, not sender

The leader is uniquely the bottleneck because it loses on all three knobs at once: it pays the consensus tax (C≈160K; the client pays 0), fans out to the most destinations (D=followers vs 1), and frames the largest unit (a log entry > a raw msg, so fewer msgs/datagram). The client and follower have friendlier values in the same formula. Two caveats: (1) this is the send path only — each role also has a receiver with its own limits, and the follower is bound by its receive path (ingesting the log), so the sender formula isn’t its ceiling at all; (2) M_pkt differs by role because framed size differs.

Peak TPS is NOT latency

The formula gives a throughput ceiling (a rate). It says nothing about per-message latency. Latency is governed by ρ = offered_load / peak_TPS, in three regimes:

regime	ρ	latency	tail
Safe	ρ < ~0.7	≈ fixed pipeline transit (metal leader ~150µs; large VM ~5ms)	tight SD ✅
Knee	~0.7 < ρ < ~1.0	queueing delay climbs steeply toward the ~220ms cap	tail blows out, SD widens ❌
Saturated	ρ > 1.0	flat at buffer_window / drain ≈ 220ms, every msg	uniform, deterministic ⚠️

• Below the knee, latency is independent of throughput. Being far under the ceiling doesn’t make messages faster — latency is the sum of fixed stage costs (client→ingress, ingress→append, append→wire, follower ack ~25µs, commit). The peak only matters through ρ.
• The ceiling touches latency only via ρ. As ρ→1, queueing delay ≈ service_time · ρ/(1−ρ) blows up, capped by the bounded log window at termLength/2 ÷ drain ≈ 220ms.
• “Controllable within an SD” is only honest in the safe regime. As ρ approaches 1 the tail runs away from the mean, so mean/SD stop describing the latency you actually see. Design to ρ < ~0.7, not “anything under peak.”

Example — same 128B message, latency differs ~1000× purely from ρ: 128B/8192 @ 800K is ρ=0.13 → ~150µs; 128B/1408 @ 800K is ρ≈1.0 → ~173ms.

The MTU relationship

MTU is the msgs-per-syscall multiplier (M_pkt) in the formula — the cheapest lever on the sender ceiling. Raising 1408→8192 lifts M_pkt ~6× (128B: 8→50), so the same syscall budget carries ~6× the load → ρ drops ~6× → clears the knee.

Proven: at 128B / 800K, MTU1408 sits at ρ≈1 with a tail pinned at ~173ms; MTU8192 drops ρ to ~0.13 and is always clean, tp999 ~150–320µs, 0% saturated. MTU 8192 is the max valid value here (Aeron requires MTU ≤ termLength/8; internal 64k IPC channels cap at 8192 — 8896/9000 are rejected).

Should you switch to 8K if MTU1408 already meets the requirement?

It hinges on one gating question: is jumbo (NIC MTU ≥ 9001) reliably supported on every hop?

	MTU 8192
Pros	~6× headroom (ρ → deep-safe, insures against load growth and the knee); slightly lower & steadier tail even below the knee (fewer syscalls → less sender contention); lower sender CPU. No low-load latency penalty — Aeron sends what’s available up to the MTU, it does not wait to fill a packet.
Cons	Requires jumbo end-to-end — any hop clamped to 1500 → fragmentation or a silent black-hole (DF drops); a lost 8K packet loses ~50 msgs vs ~8 (worse on lossy links; moot on a 0-loss same-VPC path); more config surface (Aeron MTU + OS/NIC MTU must agree or you get silent fragmentation).

Decision rule:

• Jumbo reliably supported (AWS same-VPC / cluster placement group) or ρ already near the knee (0.7–0.9) → switch to 8K (low-risk headroom + tail win).
• ρ < 0.5 at 1408 and jumbo support uncertain / crosses VPC-peering / TGW / VPN / on-prem / lossy → stay at 1408 (simpler, no black-hole risk; 8K gains are marginal when already safe).

See AWS Hard Limits for what jumbo support you can and can’t rely on.

Raising the ceiling itself (when M_pkt isn’t enough)

If you need more than MTU can give, raise S (the send path), ranked by effort:

1. sleep-ns / backoff idle on cluster + archive — coalesces consensus control packets, cutting C (~160K → 75K) and freeing ~12% of the budget. Config-only, partial. See Smart Batching and Idle Strategy.
2. MTU 8192 — the M_pkt multiplier. Config-only (needs jumbo). The primary fix.
3. Message batching — more app payload per cluster message → higher effective msgs/packet.
4. DPDK media driver — kernel-bypass, poll-mode, burst TX replaces the ~2.5µs sendmsg with a ~50–150ns userspace ring write → S from ~400K to ~5–15M+/s (10–30×). Heavy ops (hugepages, NIC binding); reserve for multi-million-TPS needs. Like MTU, it only removes the sender ceiling — ingress-window / bandwidth / append-thread then bind. See Media Driver Selection: Java vs C.

TL;DR

• peak_TPS = (S−C)/D × M_pkt, with S ≈ 400K sendmsg/s on one thread (Intel m7i). Retrodicts all runs.
• MTU is the M_pkt multiplier — the cheapest way to raise the ceiling; 8192 clears 128B/800K.
• Peak ≠ latency. Below ρ≈0.7, latency = fixed pipeline transit (tight SD); 0.7–1.0 is the knee where the tail runs away toward ~220ms; > 1.0 pins flat at ~220ms. Operate at ρ < 0.7.
• Switch to 8K if jumbo is reliable or you’re near the knee; stay at 1408 if already safe (ρ < 0.5) and jumbo support is uncertain.
• More cores / a bigger box do not help (single thread). Only M_pkt (MTU / batching) or S (DPDK).

This is the capacity-and-latency lens; its close companion is Sender Queueing and Burst Pacing (the same single Sender thread, seen as a burst-drain queue), and the receiver-side view is the micro-burst model.