Benchmarking Micro-Bursts

Most Aeron benchmarks measure steady-state throughput: constant N messages/second over a sustained period. This model is useful for sizing, but it misses the failure mode that actually pages people in production — the micro-burst.

What is a micro-burst?

A micro-burst is a short spike of traffic that arrives faster than the average rate, typically lasting 1–5 ms. In matching-engine workloads this is the norm, not the exception:

• A market data tick triggers 50K order updates in < 1 ms
• A fill cascade propagates across multiple ME shards simultaneously
• A recovery replay dumps buffered events at wire speed

The system handles the average rate fine. It’s the peak instantaneous rate — measured in messages per millisecond — that overflows buffers and creates latency spikes.

Note

This page is the receiver-side view (buffer overflow → drops → NAK recovery). The same unpaced burst also builds a queue on the sender before it ever hits the wire — see Sender Queueing and Burst Pacing for the closed-form model and how to pace dispatch against a latency target.

Why steady-state benchmarks lie

Metric	Steady-state benchmark	Production reality
Rate	400K msg/s (even)	0 → 50K in 1 ms, then idle
Buffer pressure	Smooth, never spikes	8.5 MB arrives in < 1 ms
p99	Looks great	20 ms spikes once/second
UDP drops	Zero	Hundreds of thousands

A steady-state 400K msg/s benchmark spreads packets evenly — ~400 per ms. But the production burst delivers 50K in 1 ms — 125× the instantaneous rate. This is the gap that kills.

The key metric: peak messages per millisecond

For micro-burst benchmarking, the primary metric is:

Peak msg/ms = maximum messages delivered within any 1 ms window

This determines the minimum buffer chain depth required to survive without drops.

Derived requirements

Given peak msg/ms = P and message size = S bytes:

Buffer layer	Minimum size
ENA RX ring	P × (number of ms to survive) entries
SO_RCVBUF (kernel socket)	P × S × burst_duration_ms
Aeron flow-control window	≥ P × S × RTT_ms
Term buffer	≥ 2 × P × S × max_stall_ms

If any layer in this chain is undersized for the peak rate, packets drop and NAK recovery adds 10–20 ms to the tail.

Designing a micro-burst benchmark

1. Sender model

Don’t use a constant-rate sender. Instead, model the production burst pattern:

// Micro-burst model: send BURST_SIZE messages as fast as possible,
// then sleep for INTERVAL_MS before the next burst.
final int BURST_SIZE = 50_000;       // messages per burst
final int INTERVAL_MS = 1_000;       // 1 burst per second (worst case)
final int MESSAGE_LENGTH = 170;      // bytes

while (running) {
    // Burst phase: send at wire speed (no pacing)
    for (int i = 0; i < BURST_SIZE; i++) {
        BUFFER.putLong(0, sequence++);
        BUFFER.putLong(8, getNowUs());
        while (publication.offer(BUFFER, 0, MESSAGE_LENGTH) < 0) {
            idleStrategy.idle();
        }
    }

    // Idle phase: simulate inter-burst gap
    LockSupport.parkNanos(INTERVAL_MS * 1_000_000L);
}

2. Receiver measurement

On the receiver, measure:

• Burst absorption time — how long from first to last message in the burst
• Max instantaneous rate — sliding 1 ms window message count
• UDP buffer errors — netstat -su | grep "receive buffer errors" must not climb
• Latency distribution — record nowUs - sendTimeUs for every message

# Monitor during run (must be FLAT):
watch -n1 "netstat -su | grep 'receive buffer errors'"

3. What to vary

Parameter	Range	Why
Burst size	10K – 200K	Find the overflow threshold
Message size	88 – 1400 bytes	Larger messages overflow faster
Number of senders	1 – 8	Simultaneous bursts compound
ENA RX ring	1024 – 16384	Find minimum safe value
SO_RCVBUF / rmem_max	208KB – 256MB	Prove the kernel clamp
aeron.rcv.initial.window.length	128KB – 32MB	Flow-control window

4. Pass/fail criteria

A burst configuration passes if:

• Zero UDP buffer errors during the entire run
• p99 latency < 1 ms (no NAK-driven spikes)
• No Aeron loss-gap-fill or flow-control back-pressure events

A burst configuration fails if:

• UDP buffer errors climb (buffer overflow)
• Periodic latency spikes appear (NAK recovery)
• ethtool -S shows rx_queue_N_rx_drops > 0

Real-world example

From a production matching-engine workload:

• 8 senders × 50K msg/s (average 400K total)
• Burst pattern: each sender delivers its 50K in ~1 ms, then idles
• Peak instantaneous: 50K msg/ms per sender
• Message size: 170 bytes

With default rmem_max (208KB): 50K × 170B = 8.5 MB arrives in 1 ms. The 208KB socket buffer overflows instantly → 20 ms NAK recovery spikes once per second.

With rmem_max = 128MB: the kernel buffer absorbs the entire burst, zero drops, p99 < 200 µs.

Relationship to the buffer calculator

The Buffer Sizing Calculator sizes buffers for worst-case burst scenarios. Enter your peak burst rate as the TPS (not the average), and set the stall headroom to your burst duration. The calculator will output the minimum buffer chain for zero-drop operation.

Summary

	Steady-state benchmark	Micro-burst benchmark
Primary metric	Sustained msg/s	Peak msg/ms
Failure mode tested	Throughput saturation	Buffer overflow
Production relevance	Capacity planning	Latency tail
Key parameter	Term buffer, window	SO_RCVBUF, rmem_max, ENA ring