Work

Exactly-Once Delivered Zero Duplicates at 0.634 Times the Throughput: A Pre-registered Fault-Injection Study of Kafka Delivery Semantics on a Single Node

Year: 2026
Type: preprint
Abstract: Kafka's delivery semantics are usually quoted from documentation: at-least-once may duplicate, exactly-once should not, and an acknowledged write on a healthy log should survive. This study counted them instead. Under a pre-registration locked before any measured run, a three-stage pipeline (source, processor, verifier) moved 20,000 sequenced 256-byte messages through a single-partition topic pair on a single-node Apache Kafka 3.9.0 broker, in two configurations: A (at-least-once: acks = all, no idempotence, auto-committing consumer) and B (exactly-once: idempotent source, read-committed consumer, transactional processor committing output writes and offsets together in batches of at most 100). Four scenarios (baseline, consumer kill, broker kill, rebalance kill) ran 3 times each, 24 runs, with correctness scored at the output topic from sequence numbers alone. All three frozen hypotheses were CONFIRMED. H1: configuration B produced zero duplicates and zero losses in all 9 of its fault runs, while configuration A produced duplicates in all 3 of its consumer-kill runs (5,141, 8,805 and 5,349 duplicate records out of 20,000, a quarter to nearly a half of the stream). H2: exactly-once costs throughput, median baseline pipeline throughput 3,337.8 messages per second for B against 5,264.5 for A, a ratio of 0.634, below the frozen 0.80 line. H3: no acknowledged message was lost in any of the 6 broker-kill runs, with median broker recovery of 22,406.5 ms; the claim's scope is one node and one disk, stated in the registration itself. Two cells are degraded and the paper prices them openly: the configuration A rebalance kill never landed (the stream finished inside the frozen 1.0 s window in all 3 runs), and the configuration B broker kill landed after its fast source had finished, so it stress-tested the transactional processor mid-stream rather than the source. The trade the documentation describes is real, and its measured terms on this machine are: all of the duplicates or none of them, at about 63 percent of the throughput.
Language: English
Open Access status: green
OpenAlex ID: W7235996732
Cites:
Cited by:
Related to: 10