Performance Tuning

Tuning Aeron is the art of pulling three levers at once — p50, p99, and throughput — knowing that almost every knob trades one against another. This section is the practical playbook.

The one thing to remember

p99 is always about avoiding variance — cache misses, queue depth, context switches, NUMA penalties. p50 is about reducing baseline cost. Throughput is about keeping the pipe full. The hardware-level levers (NUMA, cache, kernel bypass, CPU pinning) have the most dramatic effect on tail latency.

In this section

• Step-by-Step Tuning Methodology — A practical workflow: sensible defaults, message-size validation, incremental load testing, symptom-driven tuning, and the L3 cache sizing rule.
• Transport Performance Factors and Their Impact — The ten levers that affect Aeron Transport performance and how each maps to throughput, p50, and p99/tail latency.
• Throughput & Latency Model — A predictive model for an Aeron® Cluster's peak TPS, the MTU relationship, and why peak throughput is not latency — derived from the driver's single-Sender-thread bottleneck and validated against benchmark runs.
• Bandwidth Delay Product (BDP) — How BDP determines minimum buffer sizing for sustained throughput across CPG, same-AZ, cross-AZ, and cross-region round-trip latencies.
• Parameter Reference — Every major Aeron transport parameter and exactly how it pulls throughput, p50, and p99 — with guidance on how to set it.
• The Four Receive-Path Buffers — The four nested buffers a packet traverses on the receive path — ENA RX ring, OS socket buffer, Aeron term buffer, and the flow-control window — their constraints, and how to size them as a consistent stack.
• Buffer Sizing Calculator — Interactive calculator for Aeron receive-path buffer sizing — computes initial window, term buffer, SO_RCVBUF, NAK timers, and more from your traffic profile.
• NAK Timer Tuning (Lever B) — How to derive aeron.nak.multicast.max.backoff, group.size, and retransmit linger from RTT and receiver count — with measured results: a 79× tail collapse under seeded loss.
• Sender Queueing and Burst Pacing — A multi-millisecond latency tail that appears with zero packet loss and zero back-pressure — caused by the sender draining a burst faster than its own media driver can serialize it. The closed-form model and how to pace against a latency target.
• Smart Batching and Idle Strategy — Why application-level batching is an anti-pattern: the C media driver coalesces sends via sendmmsg, and the idle strategy is the real latency-vs-CPU knob.
• Core Isolation & Thread Pinning — GRUB isolcpus/nohz_full vs taskset vs numactl — what each controls, how they compose into one isolation recipe, and a worked Aeron media-driver example.
• NUMA, Memory Hierarchy, and Cache Locality — Why mechanical sympathy matters: dual-socket NUMA layout, L1/L2/L3/DRAM latencies, QPI cost, and keeping the term buffer inside L3 cache.
• Host & OS Latency Parameters — The system layer beneath Aeron's transport knobs — CPU C-states/P-states, frequency pinning, SMT, IRQ affinity, transparent hugepages, socket buffers and the JVM flags that move the tail. Each parameter with its effect on p50/p99/throughput, where it's set, whether it survives reboot, and how to revert. Includes how to detect and stress-confirm a flat all-core clock.
• AWS Hard Limits You Cannot Tune Past — PPS ceilings, per-flow caps, bandwidth, the utilization-latency headroom rule (~60% on a hot path), and BDP-driven buffer minimums on EC2.
• Protocol Encapsulation: TCP vs Aeron®/UDP — Layer-by-layer byte overhead of the TCP/IP stack compared with Aeron's leaner UDP plus NAK-based reliability protocol.
• When You Need the DPDK Media Driver (and When You Don't) — The kernel-bypass DPDK driver removes the packet-rate ceiling that saturates the Java/C drivers — but it costs dedicated cores, hugepages, a bound ENI, and has a silent misconfiguration footgun. A decision guide: the four cases that justify it, the cases that don't, and why the driver is rarely your bottleneck.
• Loss Injection & CUBIC Congestion Control — Two production levers from the driver's io.aeron.driver.ext package: inject deterministic packet loss for chaos testing, and swap the static receiver window for the CUBIC congestion controller on shared or long-fat links.
• Aeron® Cluster Snapshot Performance: A Matching-Engine Case Study — From a snapshot that failed above 16 resting orders to a 5M-order take in 179 ms under ZGC, then a 3-node cluster where the Archive's per-block fsync, not encoding, sets the client pause.

Where to start

1. Get threading mode and idle strategy right (DEDICATED + busy-spin for Sender/Receiver).
2. Isolate and pin cores so the OS scheduler never preempts Aeron’s agents.
3. Size buffers and windows to your BDP.
4. Then measure — see Benchmarking — before reaching for kernel bypass.

Note

For why these knobs exist at the protocol level, see The Aeron Files. This section is about what to set and why it moves the numbers.