#producer
Posts
-
Digging into Kafka's Source 5: Inside the Producer — From send() to Batches and the Sender Thread
KafkaProducer.send() sends nothing over the network. It serializes the record, picks a partition, appends it to a per-partition batch in the RecordAccumulator and hands back a Future; the one thread that actually sends is kafka-producer-network-thread. We follow in the source the if in ready() where linger.ms and batch.size apply, the built-in partitioner that sticks keyless records to one partition, the BufferPool that hands out batch memory and max.block.ms, retries and delivery.timeout.ms, and max.in.flight.requests.per.connection, then check them with examples that count batches with kafka-dump-log.sh and print which thread runs the callback.