Benchmarking

A benchmark you can’t trust is worse than no benchmark. This section covers how to measure Aeron honestly, the traps that produce flattering-but-fake numbers, and the published results to calibrate against.

Coordinated omission will lie to you

The single most common benchmarking mistake: measuring only the requests that the system let you send. Under load, a stalled system sends fewer requests, so the slow ones never get counted — and your p99 looks great. Use a constant-rate load generator and record latency against the intended send time, not the actual one.

In this section

• Methodology — open vs. closed loop, warmup, steady-state, and reproducibility.
• Latency with HdrHistogram — capturing the full distribution, not just an average.
• Throughput tests — finding the sustainable rate, not the bursty peak.
• Published results — Transport and Cluster numbers at 100K → 5M msg/s.

A minimal honest benchmark

1. Warm up the JVM and let the JIT settle (see JVM warmup).
2. Drive a fixed message rate well below saturation; record latency per message.
3. Capture into HdrHistogram; report p50 / p99 / p99.9 / max — never the mean alone.
4. Repeat across runs; report variance, not a single lucky run.

Tip

Always benchmark the C media driver and Java media driver separately — their tail behavior differs significantly. See the media driver comparison.