#rebalance
Posts
-
Digging into Kafka's Source 11: The Consumer and the Group Coordinator — Inside poll() and Rebalancing
KafkaConsumer is a thin shell that hands every call to either ClassicKafkaConsumer or AsyncKafkaConsumer, depending on group.protocol. With the default, classic, the thread that calls poll() runs the rebalance itself through JoinGroup and SyncGroup, and a leader consumer computes the assignment. With the consumer protocol (KIP-848), a background thread talks to the broker with a single ConsumerGroupHeartbeat request and the broker decides the assignment. We follow both, then the broker-side group coordinator and its CoordinatorRuntime, which writes group state to __consumer_offsets as records and rebuilds it by reading them back — checked with an example that adds two consumers one after the other, and with kafka-dump-log.sh.